Running Alluxio Locally

Requirement

The prerequisite for this part is that you have Java (JDK 7 or above).

Download the binary distribution of Alluxio 1.4.0:

$ wget http://alluxio.org/downloads/files/1.4.0/alluxio-1.4.0-bin.tar.gz
$ tar xvfz alluxio-1.4.0-bin.tar.gz
$ cd alluxio-1.4.0

To run in standalone mode, make sure that:

  • Set alluxio.master.hostname in conf/alluxio-site.properties to localhost (i.e., alluxio.master.hostname=localhost).

  • Set alluxio.underfs.address in conf/alluxio-site.properties to a tmp directory in the local filesystem (e.g., alluxio.underfs.address=/tmp).

  • Remote login service is turned on so that ssh localhost can succeed. To avoid the need to repeatedly input the password, you can add the public ssh key for the host into ~/.ssh/authorized_keys. See this tutorial for more details.

Format Alluxio Filesystem

This step is only required when you run Alluxio for the first time. All previously stored data and metadata in an Alluxio filesystem on this server will be erased.

$ ./bin/alluxio format

Start Alluxio Filesystem Locally

Start Alluxio with sudo privileges

By default, on startup Alluxio will create a RAMFS as its in-memory data storage. This step requires sudo privileges to perform “mount”, “umount”, “mkdir” and “chmod” operations. There are two approaches to achieve this:

  • Start Alluxio by a superuser, or
  • Give limited sudo privileges to the running user (e.g., “alluxio”) by adding the following line to /etc/sudoers on Linux: alluxio ALL=(ALL) NOPASSWD: /bin/mount * /mnt/ramdisk, /bin/umount * /mnt/ramdisk, /bin/mkdir * /mnt/ramdisk, /bin/chmod * /mnt/ramdisk This allows Linux user “alluxio” to mount, umount, mkdir and chmod (assume they are in /bin/) a specific path /mnt/ramdisk with sudo privileges without typing the password, but nothing else. See more detailed explanation about Sudoer User Specifications.

With the proper user, run the following command to start Alluxio filesystem.

$ ./bin/alluxio-start.sh local

Start Alluxio without sudo privileges

Alternatively, if a RAMFS (e.g., /path/to/ramdisk) is already mounted by the system admin, you can specify the path in conf/alluxio-site.properties:

alluxio.worker.tieredstore.level0.alias=MEM
alluxio.worker.tieredstore.level0.dirs.path=/path/to/ramdisk

and start Alluxio without requiring sudo privileges:

$ ./bin/alluxio-start.sh local NoMount

Verify Alluxio is running

To verify that Alluxio is running, you can visit http://localhost:19999, or see the log in the logs folder. You can also run a sample program:

$ ./bin/alluxio runTest Basic CACHE THROUGH

For the first sample program, you should be able to see something similar to the following:

2015-11-20 08:32:22,271 INFO   (ClientBase.java:connect) - Alluxio client (version 1.0.0) is trying to connect with FileSystemMaster master @ localhost/127.0.0.1:19998
2015-11-20 08:32:22,294 INFO   (ClientBase.java:connect) - Client registered with FileSystemMaster master @ localhost/127.0.0.1:19998
2015-11-20 08:32:22,387 INFO   (BasicOperations.java:createFile) - createFile with fileId 33554431 took 127 ms.
2015-11-20 08:32:22,552 INFO   (ClientBase.java:connect) - Alluxio client (version 1.0.0) is trying to connect with BlockMaster master @ localhost/127.0.0.1:19998
2015-11-20 08:32:22,553 INFO   (ClientBase.java:connect) - Client registered with BlockMaster master @ localhost/127.0.0.1:19998
2015-11-20 08:32:22,604 INFO   (WorkerClient.java:connect) - Connecting local worker @ /192.168.2.15:29998
2015-11-20 08:32:22,698 INFO   (BasicOperations.java:writeFile) - writeFile to file /default_tests_files/BasicFile_CACHE_THROUGH took 311 ms.
2015-11-20 08:32:22,759 INFO   (FileUtils.java:createStorageDirPath) - Folder /Volumes/ramdisk/alluxioworker/7226211928567857329 was created!
2015-11-20 08:32:22,809 INFO   (LocalBlockOutStream.java:<init>) - LocalBlockOutStream created new file block, block path: /Volumes/ramdisk/alluxioworker/7226211928567857329/16777216
2015-11-20 08:32:22,886 INFO   (BasicOperations.java:readFile) - readFile file /default_tests_files/BasicFile_CACHE_THROUGH took 187 ms.
Passed the test!

And you can visit Alluxio web UI at http://localhost:19999 again. Click Browse in the navigation bar and you should see the files written to Alluxio by the above test.

To run a more comprehensive sanity check:

$ ./bin/alluxio runTests

You can stop Alluxio any time by running:

$ ./bin/alluxio-stop.sh all
Need help? Ask a Question