PLANET
  • NEWS
  • PEOPLE
  • ISSUES
  • GITHUB ACTIVITIES
  • GITHUB STATISTICS
  • Proudly hosted by GitHub

gsoc-2014

GSoC 2014: Dockerizing Flumotion, Building a Build System

Mar 13, 2014 • Max (Dockerization)

Tags: news dockerization

Share:

Proposer: Max Stritzinger

University of California, Santa Barbara
[email protected]
CARAM on freenode

Synopsis

This is a proposal for TimVideos for GSoC 2014. This proposal is centered around the 'Dockerization' of Flumotion allowing for quick and portable deployments of the TimVideos streaming system. Following completion of Dockerizing the streaming system, work will begin on an automated build tool based on the newly Docker'd components.

~Dockerization~

Docker is an open-source framework allowing for the creation of portable application containers. Docker allows for an application and all of its dependencies to be wrapped in one of these containers and deployed with ease to any host with Docker installed. My proposal is to wrap the streaming system up in a few Docker packages to make it extremely easy and quick to deploy. For example, following 'Dockerization', setting up the website portion of the streaming system might be as simple as: docker pull streaming-system/website
docker run -name website -p 80:80 -d streaming-system/website python streamingsystem/tools/serverstart.py someconf_file

Automated Build System

Dockerization would be the meat of the project, however my belief is that time will allow for development of a script/tool to generate most of the streaming system automatically assisted by Docker. Following delivery of the Docker portion of the proposal, I would plan to continue working on an automated build/deployment tool. Docker fits in really well with an automated deployment tool. Docker supports running in daemon mode, so you can imagine PXE/AWS booting an image running a Docker daemon. Then, deployment and configuration for our sample website component becomes as simple as:
docker -H remotehost:4243 pull streaming-system/website
docker -H remotehost:4243 run -name website -p 80:80 -d streaming-system/website python streamingsystem/tools/serverstart.py someconf_file You can imagine the ease at which this can be scripted out!

My Proposed Work

  1. Starting with a base Ubuntu Docker image, I will build a streaming-system/base Docker image with all dependencies (including watchdog, register) installed. This image will be capable of functioning as any of the three components, given some additional configuration
  2. Building off of the streaming-system/base package, I will build separate images for the Encoder, Collector, and Website components of streaming system. These images will come preconfigured for their specific component function and will be able to accept configuration arguments on startup for easy setup. This will involve generation of custom scripts/config files for each of the different components.
  3. streaming-system/(base, website, encoder, collector) will published to the docker.io index under the streaming-system or similar username
  4. Following the completion of the Docker portion of the project, I will then begin work on the automated build and deployment system.
  5. The automated build system will be integrated around a Cobbler daemon and the AWS Python API.
  6. The build tool will, based on the configuration file/utility, boot any PXE images it needs via cobbler and start EC2 instances via AWS, push the correct component(s) to each host (through a docker remote pull), and then start each docker instance based on the configuration options specified.

Timeline

  • Now - May 18th : Prior to the official start of the program I'll be active on IRC and poking around the streaming system components to gain familiarity with the different parts of the streaming system.
  • May 19th - May 25th: Development environment setup, setup of the streaming-system/base container.* May 26th - June 1st: Setup / Configuration of the streaming-system/website container. By the end of this week we will be able to 'docker pull' a website container and start the web service with a simple configuration file and start script.
  • June 2nd - June 11th: Setup / Configuration of the streaming-system/collector container. By the end of this week we will be able to 'docker pull' a collector container and start the encoding service with a simple configuration file and start script
  • June 12th - June 22nd: Setup / Configuration of the streaming-system/encoder container. By the end of this week we will be able to 'docker pull' an encoder container and start the collecting service with a simple configuration file and start script
  • June 23rd - June 29th: Dockerization should be fully complete! Work begins on the automated build system. By the end of this week we will have a tool that will take a component type, some parameters regarding it's associated components, and will output a configuration file.
  • June 30th - July 13th: Extend build tool (or possibly create a separate 'boot' tool) to incorporate host booting. We will be able to take a list of hosts and their types(EC2, PXE) and deploy a docker-enabled image to them.
  • July 14th - August 10th: Add support to our tool for specifying entire systems of components with the hosts they will run on. Tool will boot hosts, deploy docker containers to them, and start the components. This should correctly start the streaming system
  • August 11th - August 18th: Clean up documentation, final testing/debugging and debrief to TimVideos team.

Accountability

I am very much interested in completing my project successfully as I am sure the TimVideos team is. Some accountability is a great way to make sure that happens! I am (and will be) almost always available on IRC for interrogation and would propose bi or tri-weekly updates/progress meetings with my mentor(s) to ensure that I'm on the right track. In addition, I would plan on keeping a blog tracking my work throughout the project on my website.

About Me

  • Second year student at UC Santa Barbara, studying Biochemistry and Computer Science. I currently hold a 3.89 GPA
  • Student Web/App developer for Associated Students at UCSB.
  • Very available this summer (the second week of June I have one or two finals, but I'm free otherwise)
  • Familiar with many languages and web technologiesincluding, but not limited to:
    • AWS
    • Python
      • Flask
    • Haskell
    • SQL (experienced with MySQL specifically)
    • PHP
    • Wordpress
    • General Bash/Shell scripting
    • (Somewhat rusty) DevOps/SysAdmin experience with:
      • CentOS/RHEL
      • Ubuntu Server
      • MySQL Server
  • I am, of course, available for contact almost anytime on IRC or through e-mail as listed above.

Comments, Concerns?

If there's anything you'd like to share, feel free to post it below or give me a ping on IRC. [Docker]: https://www.docker.io/

Originally posted on Max S