Note: OpenShift Origin has recently moved to use Ruby 1.9 and Fedora 17. Following instructions on this post will no longer lead to a working Origin image. I am working on a new set of instructions of the latest release of Origin.
I have seen a growing interest in setting OpenShift Origin on OpenStack machines. This post will walk you through the basic setup of building and running OpenShift Origin on OpenStack.
Setting up OpenStack
The first step to get started is setting up OpenStack on a Fedora 17 machine. The folks over at Fedora have already put together an excellent how-to to get you started. After the installation is completed, you should have a OpenStack dashboard available at http://localhost/dashboard
A few things to note about the process:
I started from a Fedora 17 net-install disk and chose “Software development” which gave me a gnome-ui and basic software dev tools. It should be possible to do the intal with a minimal install as well.
I also ran into qpid-cpp-server issues where F17 reported it installed but the binaries were not present. I had to uninstall the qpid-cll-client and yum install qpid-cpp-server-0.16-1.0.16-fc17.2.x86_64. The base issue seems to be a multi-version conflict around i686 vs x86_64. This seemed to solve the problem.
Note: This does not seem to be an issue anymore.
I also found it handy to have virt-manager installed to debug the setup and build new images.
If you are running OpenStack on a physical machine, you will need to open the /etc/nova/nova.conf file and remove libvirt_type=qemu
Option 1: Building the OpenShift image using Oz
Install Oz (You may need to enable the updates-testing repository to resolve dependencies)
|
1 |
yum install -y --enablerepo=updates-testing oz |
Create a new template file named openshift.tdl (based on f16-jeos.tdl provided by Fedora)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 |
openshift_origin OpenShift Origin Fedora 16 x86_64 http://download.fedoraproject.org/pub/fedora/linux/releases/16/Fedora/x86_64/os/ <command></command> sed -i 's/rhgb//' /etc/default/grub sed -i 's/quiet/console=ttyS0/' /etc/default/grub grub2-mkconfig > /etc/grub2.cfg cat > /etc/rc.d/init.d/openstack-sshkey <<EOF #!/bin/bash # # live: Late init script for live image # # chkconfig: 345 99 01 # description: Late init script for configuring image start() { if [ ! -d /root/.ssh ]; then mkdir -p /root/.ssh chmod 700 /root/.ssh fi # Fetch public key using HTTP ATTEMPTS=10 while [ ! -f /root/.ssh/authorized_keys ]; do curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/aws-key 2>/dev/null if [ \$? -eq 0 ]; then cat /tmp/aws-key >> /root/.ssh/authorized_keys chmod 0600 /root/.ssh/authorized_keys restorecon -rv /root/.ssh rm -f /tmp/aws-key echo "Successfully retrieved AWS public key from instance metadata" else FAILED=\$((\$FAILED + 1)) if [ \$FAILED -ge \$ATTEMPTS ]; then echo "Failed to retrieve AWS public key after \$FAILED attempts, quitting" break fi echo "Could not retrieve AWS public key (attempt #\$FAILED/\$ATTEMPTS), retrying in 5 seconds..." sleep 5 fi done } case "\$1" in start) start ;; stop) ;; esac exit 0 EOF chmod +x /etc/rc.d/init.d/openstack-sshkey chkconfig --add openstack-sshkey chkconfig openstack-sshkey on chkconfig network on chkconfig NetworkManager off chkconfig sshd on service NetworkManager stop service network start cat > /etc/sysconfig/network-scripts/ifcfg-eth0 <<EOF DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=yes NM_MANAGED=no EOF sed -i 's/#RSAAuthentication yes/RSAAuthentication yes/' /etc/ssh/sshd_config sed -i 's/#PubkeyAuthentication yes/PubkeyAuthentication yes/' /etc/ssh/sshd_config mkdir -p /root/.ssh touch /root/.ssh/authorized_keys chmod og-rwx -R /root/.ssh restorecon -R /root/.ssh cat > /etc/resolv.conf <<EOF nameserver 8.8.8.8 EOF useradd builder usermod -a -G mock builder yum -y install git rubygem-rake ntp su builder -c "cd /home/builder ; git clone git://github.com/openshift/crankcase.git /home/builder/crankcase" su builder -c "cd /home/builder/crankcase" cd /home/builder/crankcase/build ; rake build_setup cd /home/builder/crankcase/build ; rake build cd /home/builder/crankcase/build ; rake install_broker |
Invoke oz-image to cerate the image:
|
1 |
sudo oz-install -d4 -u openshift.tdl |
If you are using kvm/qemu, the disk will be stored in /var/lib/libvirt/images/. We will use qemu-img to create a qcow2 compressed image.
|
1 |
qemu-img convert -O qcow2 /var/lib/libvirt/images/openshift_origin.dsk origin-base.qcow |
Finally, import the image into glace.
|
1 |
glance add name="origin-base" is_public=true container_format=bare disk_format=qcow2 < origin-base.qcow |
At this point, the OpenShift Origin disk image should be available for you to launch on OpenStack.
Option 2: Building the OpenShift image manually
Using virt-manager, start a new Fedora 16 guest (net-inst iso) and log into it using virt-manager.
Create a new user to perform the OpenShift Origin build steps and add the user to the wheel group so that you can use sudo.
|
1 2 |
useradd builder usermod -a -G wheel builder |
Switch to the build user and follow the steps from my post about build a multi-node OpenShift setup. You will only need to build the broker machine and skip the “ss-setup-broker” step.
At the end of the setup, you will have a qemu VM with the OpenShift Origin packages installed. Next we need to set up the grub and ssk-key scripts.
Edit the /etc/default/grub and remove “rhgb” and “quiet”. The final default file should look like
|
1 2 3 4 |
GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="Fedora" GRUB_DEFAULT=saved GRUB_CMDLINE_LINUX="rd.md=0 rd.lvm=0 rd.dm=0 KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd.luks=0 LANG=en_US.UTF-8" |
Run grub2-mkconfig to re-generate the Grub config.
|
1 2 3 4 5 |
[root@broker php]# grub2-mkconfig > /etc/grub2.cfg Generating grub.cfg ... Found linux image: /boot/vmlinuz-3.4.2-1.fc16.x86_64 Found initrd image: /boot/initramfs-3.4.2-1.fc16.x86_64.img done |
Installing a script to setup root ssh keys.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
cat < /etc/rc.d/init.d/openstack-sshkey #!/bin/bash # # live: Late init script for live image # # chkconfig: 345 99 01 # description: Late init script for configuring image start() { if [ ! -d /root/.ssh ]; then mkdir -p /root/.ssh chmod 700 /root/.ssh fi # Fetch public key using HTTP ATTEMPTS=10 while [ ! -f /root/.ssh/authorized_keys ]; do curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/aws-key 2>/dev/null if [ \$? -eq 0 ]; then cat /tmp/aws-key >> /root/.ssh/authorized_keys chmod 0600 /root/.ssh/authorized_keys restorecon -rv /root/.ssh rm -f /tmp/aws-key echo "Successfully retrieved AWS public key from instance metadata" else FAILED=\$((\$FAILED + 1)) if [ $FAILED -ge $ATTEMPTS ]; then echo "Failed to retrieve AWS public key after \$FAILED attempts, quitting" break fi echo "Could not retrieve AWS public key (attempt #\$FAILED/\$ATTEMPTS), retrying in 5 seconds..." sleep 5 fi done } case "\$1" in start) start ;; stop) ;; esac exit 0 EOF chmod +x /etc/rc.d/init.d/openstack-sshkey chkconfig --add openstack-sshkey chkconfig openstack-sshkey on |
Make sure network and ssh will start
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
chkconfig network on chkconfig NetworkManager off chkconfig sshd start cat < /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 BOOTPROTO=dhcp ONBOOT=yes NM_MANAGED=no EOF sed -i 's/#RSAAuthentication yes/RSAAuthentication yes/' /etc/ssh/sshd_config sed -i 's/#PubkeyAuthentication yes/PubkeyAuthentication yes/' /etc/ssh/sshd_config mkdir -p /root/.ssh touch /root/.ssh/authorized_keys chmod og-rwx -R /root/.ssh restorecon -R /root/.ssh |
To keep the final image size to a minimum, we need to make sure that all the empty space on the filesystem is zero’d out. I use zerofree to do this.
|
1 |
yum -y install zerofree |
We will need to switch to single user mode and then remount the filesystem as read-only before we can run zerofree.
|
1 2 3 4 5 |
init 1 mount -n -oremount,ro -t ext4 /dev/vda3 / mount -n -oremount,ro -t ext4 /dev/vda2 /boot/ zerofree /dev/vda3 zerofree /dev/vda2 |
If you are using kvm/qemu, the disk will be stored in /var/lib/libvirt/images/. We will use qemu-img to create a qcow2 compressed image.
|
1 |
qemu-img convert -O qcow2 /var/lib/libvirt/images/origin-base.img origin-base.qcow |
Finally, import the image into glace.
|
1 |
glance add name="origin-base" is_public=true container_format=bare disk_format=qcow2 < origin-base.qcow |
At this point, the OpenShift Origin disk image should be available for you to launch on OpenStack.
Final configuration
Once the origin machine is up and running, ssh into the machine as root and run ss-setup-broker to complete the setup process for a broker machine.
i execute “sudo openstack-db –service nova –init”,but mysql is not installed successfully,the errors as follows:
Error: Package: mysql-server-5.5.25a-1.fc17.i686 (updates)
Requires: mysql-libs(x86-32) = 5.5.25a-1.fc17
Installed: mysql-libs-5.5.27-1.fc17.i686 (@updates-testing)
mysql-libs(x86-32) = 5.5.27-1.fc17
Available: mysql-libs-5.5.23-1.fc17.i686 (fedora)
mysql-libs(x86-32) = 5.5.23-1.fc17
Available: mysql-libs-5.5.25a-1.fc17.i686 (updates)
mysql-libs(x86-32) = 5.5.25a-1.fc17
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
mysqld is not running. Would you like to start it now? (y/n): y
Failed to issue method call: Unit mysqld.service failed to load: No such file or directory. See system logs and 'systemctl status mysqld.service' for details.
error reading information on service mysqld: No such file or directory
Since this is a fresh installation of MySQL, please set a password for the 'root' mysql user.
Enter new password for 'root' mysql user:
Enter new password again:
/bin/openstack-db: line 196: mysql: command not found
Failed to set password for 'root' MySQL user.
Hi George, I’m not an expert as far as the OpenStack installation goes but it looks like you ran into a broken MySql RPM onFedora 17.
I would recommend jumping on #fedora IRC channel on free node and asking for help.
i have followed the instructions of build your own paas openshift and completed the openshift locally.
Now how can i deploy a wordpress app
and how to deploy this openshift on openstack
Hi Ravishankar. I will posting updated instructions to work with F17 soon. The current codebase runs only on Ruby 1.9 so it wont work as-is using the instructions posted above.
I would also recommend jumping on IRC #openshift-dev on freenode if you need help with getting wordpress up.
Where can we find instructions for Openshift on Openstack for fedora 17 and 18?
Thanks