Instructions for Building Your ODROID XU4 Compute Cluster

  1. Download the OS image from the hardkernel.com web site.
  2. The file comes zipped in a .xz file. Unzip the file. For Windows users, 7-Zip can unzip it. For Linux users, you can use the command unxz the_file.img.xz (info on unxz).
  3. Flash the image onto two microSD cards. For Windows users, Win32 Disk Imager works. Hardkernel provides a link labelled "OS Image flashing tool for Windows" to their version of Win32 DiskImager on their downloads page.
     
    Linux users can use dd to flash the image onto the microSD cards. If it's a brand new card, you should be able to just use the commands
    sudo dd if=/dev/zero of=/dev/sdb bs=1M count=10 (assuming that the card is sdb - dmesg can show you.)
    dd if=~/Desktop/ubuntu-15.04-mate-odroid-xu3-20150710.img | pv | sudo dd of=/dev/sdb       (Make sure to replace the ~/Desktop/ubuntu-15.04-mate-odroid-xu3-20150710.img with the path to your image file. Also, this command will prompt you for the root password but appear to keep going while you enter it. Make sure to enter the root password.) Once you've gotten your command prompt back because the commands are done, type sync. When that's done, unmount/eject the card properly through the GUI or command line and then remove it.
     
    If your card is not brand new (AKA it has been written on already), you may need to do a little extra work. Once you have flashed a card and removed it, you'll likely be unable to boot from it immediately and get an error like this: EXT4-fs (mmcblk0p2): bad geometry: block count 3856384 exceeds size of device (1003264 blocks). Put the card back in your Linux machine and you'll get a corresponding error message like this: # EXT4-fs (sdb2): bad geometry: block count 3856384 exceeds size of device (1003264 blocks) . Run the command sudo e2fsck -fy /dev/sdb2, then sync, and then eject the card properly and remove it. Now you should be able to boot from it.
  4. Once you have flashed the microSD cards, pop them into your systems.
    Set the boot switch in this image
    to boot from the microSD card.
  5. It's critical to note that the devices ship from the factory ALL using the same Mac Address. Therefore, we'll have to change the Mac Address on the systems. If you power on both before doing this, things won't work correctly, so ***do the systems one at a time beginning with this step up to and including step 13***. Connect an Ethernet cable to the system and to your network, a keyboard, mouse, and monitor/TV to the system and power it on.
  6. Once Linux has booted, log on. The username and passsword are both odroid.
  7. Run the ODROID utility program (there's an icon for this on the desktop) and resize your root partition and then reboot the system.
  8. Download and install all the software using the command sudo apt-get install gedit vim nfs-kernel-server nfs-common rpcbind nfs-client openssh-server mpich2 (source).
  9. Grab the program below which comes from Blaise Barney's MPI Tutorial on the LLNL web site and save it as mpitest.c.
  10. Get a script that will change the IP Address for you with the command wget web.centre.edu/david.toth/research/staticIpXU4.sh but don't run the script yet.
  11. Change the Mac Address of the system. As sudo, edit the /media/boot/boot.ini file. Look for the line
    setenv macaddr "00:1e:06:61:7a:39"
    and change the 39 to 40 (for the top-master node) or 41 (for the bottom-slave node).
  12. Run the staticIPXU4.sh script you downloaded two steps ago by typing sh staticIPXU4.sh 100 top-master or sh staticIPXU4.sh 101 bottom-slave.
  13. Reboot the system. You may not have Internet access now, depending on how your network is configured. If your nodes are plugged into a switch that's connected to a router like one for your home, you may still, depending on how your router handles the computers. You don't need access at this point for the node you have finished anyway.
  14. Go back and repeat the steps to get your second system up and running.
  15. At this point, all you need to do is configure NFS, communication between your nodes without passwords, and test out MPI!
  16. On both nodes, run these two commands:
    sudo mkdir /sharedFiles
    sudo chown odroid:odroid /sharedFiles.
  17. On the top-master node, run these two commands:
    echo "/sharedFiles *(rw,sync)" | sudo tee -a /etc/exports
    sudo service nfs-kernel-server restart.
  18. On the bottom-slave node, as sudo edit /etc/fstab and add this line:
    top-master:/sharedFiles    /sharedFiles    nfs
  19. On both nodes, run these commands:
    ssh­-keygen ­-t rsa       (for this command, I accept the default file location and I use an empty password)
    cd .ssh
    cat id_rsa.pub >> authorized_keys
  20. On the top-master node, run this command:
    ssh-copy-id bottom-slave
  21. On the bottom-slave node, run this command:
    ssh-copy-id top-master
  22. Reboot the master node and once it comes up, reboot the slave node.
  23. From one of the nodes, do the rest of these instructions. Copy the mpitest.c file into /sharedFiles.
  24. cd /sharedFiles and then create a file called machines in that folder which contains these two lines
    top-master:8
    bottom-slave:8
    .
  25. Compile the mpitest.c file with the command mpicc mpitest.c -o hellompi
  26. Run the compiled program with the command mpirun -n 16 -f ./machines ./hellompi
  27. You should get 16 lines of output with processes 0 to 15, which should be split between the two different nodes. Note that the lines won't be in order!
  28. When you want to write MPI programs and run them, just make your source code file in the /sharedFiles folder.