Google Summer of Code Update #3

Last week was a rather slow week. I did some work on allowing the dashboard to store and process information that does not come from the BTS. This exposed a few issues with the way we were storing and presenting information. After talking to Matt, we decided that the dashboard should deal with task objects. These tasks would contain basic information such as a name, person assigned to it, status, and possibly a link to a bug. This means that we would be able to start up a new DEX project by simply choosing a project name and creating a basic text file containing a list of tasks (package names, patches, bugs, etc).

I also have been trying to get a hold of some people who participated in the last DEX project. I am interested in getting some more feedback about what went well, problems they encountered, and tools that might have made things easier. I am also talking to some people who are currently working on starting up new DEX projects to try and make dextools align with as many workflows as possible. If you fall into either of these categories, I would really appreciate it if you contact me.

There is also a new wiki page that I created so that it is easier for people who are not interested in me or the Google Summer of Code to quickly see what dextools is about and monitor its progress. I will be adding more information over the next few days.

Currently, I am continuing to work on the dashboard and talking to people about their DEX experiences. I have also begun working on some scripts to generate graphs that will go on the dashboard. These graphs will allow us to quickly check our progress on different projects. They will also allow us to see and acknowledge individuals working on the tasks. I will add more information about these graphs to this blog and to the dextools wiki page.

irssi bansearch.pl script

As a freenode staffer and Ubuntu Operator, a frequent question that I am asked is, “Why can’t I join #channel?” or “Why can’t I talk in #chanel?”. To answer this question, I normally check the ban and quiet lists for #channel, and then I check what channel modes are applied. For small channels, this is not too bad, but for channels with very large ban/quiet lists, it can take some time. This is why I have created bansearch.pl. This irssi script will attempt to determine why a user is unable to join/talk in a channel.

Currently, the script is maintained in a Launchpad bzr branch. To get and install the script, run the following commands from your home directory.

bzr branch lp:~nhandler/+junk/bansearch
mkdir -p ~/.irssi/scripts/autorun
ln -s ~/bansearch/bansearch.pl ~/.irssi/scripts/bansearch.pl
ln -s ~/.irssi/scripts/bansearch.pl ~/.irssi/scripts/autorun/bansearch.pl

You can leave out the last command if you do not want the script to run automatically when you start irssi.

Using the script is quite simple. You can use it like this:

/script load bansearch.pl
/bansearch nick [#channel]

If you do not specify a channel, it will use the current channel. The script will currently look for bans and quiets matching the user. It has some limited extban support, but it does not support CIDR bans. It also supports looking at channel modes that might prevent the user from joining/talking. The script knows whether the user is identified, so $a extbans, $~a, and cmode +r are all supported, and the script will only warn you about them if they match the user. The script was written for freenode, so while basic ban/quiet searching might work on other IRC networks, the channel mode checks will probably not work properly.

The script is still fairly new, so there are still bugs in it. If you find any, please let me know, and I will try and patch them as quickly as possible. I am also open to any feedback about how to improve the script.

Google Summer of Code Report #1

Here is my first official Google Summer of Code report. As I still had classes, the first week of coding was rather slow. However, after my first phone call with my mentor, Matt Zimmerman, things started to pick up. I spent most of my time working on a dashboard/portal for the Debian Derivatives Exchange (DEX). We decided that the best approach to this would be to store as little information ourselves as possible. Instead, we will try to pull most of our information from other sources, such as Debian’s BTS. The main advantage to this is that package maintainers do not need to change any of their workflows, but they still get to benefit from DEX’s work. Currently, I have a python script that gets a list of usertags and bugs for debian-derivatives@lists.debian.org from the BTS. This script will end up getting run by cron. It stores the information it gets in static HTML files. Finally, JQuery and Javascript are used to present the final dashboard to the user. I created a mockup showing how this looks. I also have a functioning website that looks and acts like the mockup, but the code is not publicly available yet (see below).

DEX uses Alioth to host its git branches and website. I have spent some time trying to familiarize myself with the recent changes that have taken place on Alioth, as well as trying to figure out the best way to store and execute my script to generate the website. So far, it looks like people are still deciding on the best way to handle having websites that are updated from VCS hooks (due to things being split between vasks and wagner). I will continue to look into this, and might try using an alternate host until things settle down.

Next, I plan to work on updating the dashboard to provide a way to edit, store, and display certain extra fields of information. This is necessary for certain pieces of information that are not appropriate for adding to bugs and in a few other cases. I will also be taking some time to talk to a few people who were active contributors to the last DEX project. This will allow me to learn more about their experience, find out what changes they feel should be made to make things run smoother, and get feedback on my current Summer of Code work. Finally, although I have been posting updates on my Summer of Code work to my blog [2][3], Matt thinks that having all of this information in one place will be useful. As a result, I will be creating another wiki page about the project that will contain all of the information people need to know. This page will be updated as the summer progresses. I am also hoping to have my code available in a public VCS by the time of my next progress update so that people can check it out and provide feedback.

In addition to the dashboard, I will be working on creating a few other tools to assist with the creation and tracking of new DEX projects. I will also be coming up with a way to generate graphs to visually allow us to track our progress on projects versus time as well as show the work done by people at the individual level. This last task is part of a goal to get more people involved in the DEX project.

While we may only be a few weeks into the actual Google Summer of Code, I have already learned a lot of new things. I am looking forward to working more with Matt and other members of the community over the course of the summer.

Google Summer of Code Update #2

I’ve been spending the past few days working on a dashboard for DEX. I’ve been using Python, javascript, and jquery to create this dashboard (partially as a learning exercise). Earlier today, I started thinking that a visual mockup of the dashboard might be helpful, as it will allow me to create and revise a design based on feedback without having to touch the code. At this point, I had already written a fair bit of code and had a rough but working early version of a dashboard coded. So for the first mockup, I simply created it to match my code.

Thanks to the generosity of the folks at Balsamiq, I was able to get myself a free key to create the mockup above. As I mentioned earlier, this is what the code currently looks like, not what it will look like. Next, I’m going to be spending some time revising that mockup so I have a visual goal to work towards. I also will be cleaning up the code that I currently have and making it available in a public VCS.

Google Summer of Code Update #1

I just got off the phone with my Google Summer of Code mentor, Matt Zimmerman. Some things that we discussed were that Debian Developers are quite partial to the BTS. We agreed that using the BTS to store as much information as possible would be ideal. This would allow the Debian Maintainers to see the information that pertains to them, without them having to change their workflows to check a DEX dashboard.

Usertags are a rather useful feature that we did not utilize much when processing ancient patches. In the future, we will be applying usertags to Debian bugs that DEX is tracking so that we can easily query and track them.

We also plan to create a more useful overview/dashboard page. This page will grab a list of bugs with a certain usertag from the BTS. It will then allow people to see bugs involved in a DEX project, who is working on them, what the current status in Debian is, and what the current status downstream is. Matt mentioned possibly using jquery for this. I have not used jquery before, so I will investigate it a bit more. I also noted the transition tracker as another type of dashboard that we could base the DEX overview page on. In addition, we hope to take the graph idea a step further by modifying it to take time into account. This means that we will be able to clearly see how long a certain project is taking, predict based on our current rate how long it will take to complete, and detect periods of time where we might be slowing down. We also hope to have graphs that acknowledge people at the individual level for working on DEX projects.

During the upcoming week, I will begin working on this overview page. In addition to coding, this will involve doing some more research to learn more about some of the tools and resources available to us. I hope to have some actual code to point people to by the time I write my next blog update.

Hello Planet Debian

Hello everyone. Per tradition, I will use my first post to Planet Debian as a bit of an introduction.

My name is Nathan Handler. I am Ubuntu Developer and a member of freenode staff. Currently, I am participating as a student in the Google Summer of Code program with Debian. I am working on Backend Tools and Infrastructure for DEX (aka “dextools”) with Matt Zimmerman as my mentor.

To quote the project description:

DEX is a new program designed to help improve Debian and its derivatives by merging in changes made downstream and encouraging discussions between the various projects. As this is a new project, most of the infrastructure does not exist (or is rather hackish and incomplete). This project will create the necessary backend tools and infrastructure so that all Debian derivatives can easily make use of the DEX project.

While I have not been great with updating my blog in the past, I am going to try and produce regular posts about my Summer of Code work. If you have any questions about DEX or are interested in getting involved, please check out the “Get Involved” section on the website.

dvd-slideshow

For a while now, I have been attempting to use the various popular video editing applications to create basic photo slideshows with music. However, I have recently discovered a tool called dvd-slideshow. This blog post is designed to serve mainly as a reminder and reference for myself by documenting what I did, but I hope that other people will also find it useful.

dvd-slideshow is available in the repositories, and you can install it with:

sudo apt-get install dvd-slideshow

First, you will want to use dir2slideshow (installed with dvd-slideshow) to add all of your photos to a txt file. The syntax for this command looks like this:

dir2slideshow -n 'My Slideshow' -t 3 -c 1 /home/me/Pictures

This will create a basic slideshow called ‘My Slideshow’ that contains all of the images in the /home/me/Pictures directory. Each image will be displayed for 3 seconds, and there will be a 1 second crossfade between pictures.

Adding music to the slideshow is simple. For my slideshows, I like to have a playlist of music playing in the background for the duration of the show. I don’t really care when they start and end, I just want them to play one after the other.

To do this, I added a few lines at the top of my txt file before any of my pictures.

/home/me/Music/song1.mp3:1:fadein:1:fadeout:1
/home/me/Music/song2.mp3:1:fadein:1:fadeout:1
/home/me/Music/song3.mp3:1:fadein:1:fadeout:1

This will cause song1.mp3, song2.mp3, and song3.mp3 to play in that order. The ‘1’ in the middle refers to audio track 1 (there are two audio tracks in case you want to include narrations on top of the music). I then have quick fadein/fadeout effects that each last 1 second.

I found that the volume levels of my audio files were quite different. To fix this, I installed mp3gain from the repositories and ran it with the -r option against all of my audio files.

When I created my first slideshow, I found that some of my pictures were rotated sideways even though they appeared correctly on my computer. To fix this, I modified my txt file to instruct dvd-slideshow to rotate the troublesome pictures:

/home/me/Pictures/badrotation.png:3::rotate:270

That will cause badrotation.png to be rotated 270 degrees clockwise. The ‘::’ is because I left the optional subtitle parameter blank. You can include some text between the colons if you want a subtitle to be displayed with the image.

Once you have your txt file prepared the way you want, you need to create your video file.

dvd-slideshow -n 'My Slideshow' -f 'My Slideshow.txt'

This will take your txt file and create a vob file called ‘My Slideshow.vob’. This vob file can be played in most Ubuntu media players, giving you the chance to preview your work.

At this point, I chose to use DeVeDe to create my menu and disc structure rather than the dvd-menu utility. My main reason for this was that I had used DeVeDe before and it worked quite well. I also did not want to have to convert the other video files that were being included on the disk to mpeg2. DeVeDe allowed me to create a title and simply specify the path to my .vob file.

You may have noticed that I mentioned that DeVeDe was being used to create my disc structure rather than an iso. The reason for this was that I wanted to create a DVD that people could play in a standard DVD player, but I also wanted to allow them to save copies of the pictures that were in the slideshow to their computer. DeVeDe created AUDIO_TS and VIDEO_TS folders. I simply dragged and dropped these folders to my DVD in nautilus. I then dragged and dropped my picture folder to the disc. After hitting the write button, I was left with a hybrid video and data dvd.

If you think you might be interested in using dvd-slideshow for your own project, I suggest that you check out their website. They have lots of great documentation and example files to help you get started.

SSH Key Support Added to cr-48

A while ago, I was lucky enough to receive one of Google’s cr-48 laptops to test out. I decided not to flip the switch to put it into developer mode (for several reasons). This meant that I was stuck with a rather locked-down terminal. Luckily, it had minimal ssh support, so I was able to connect to my other computer to run programs like irssi.

Recently, Google rolled out an update for Chrome OS. One of the changes contained in this update had to do with ssh. Now, simply running ‘ssh ‘ is not sufficient to connect. This will instead drop you into an ‘ssh>’ interactive prompt. At this prompt, you can simply type ‘connect’ to continue connecting. This interactive ssh program adds some additional functionality as well. Below is the output of the ‘help’ command.

crosh> ssh
ssh> help
connect - connect
dynamic-forward port - dynamic socks proxy (-D)
forward port:host:port - static port forward (-L)
help - this
host - remote hostname
key - sets private key to use (-i)
nocmd - don't execute command (-N)
port - port on remote host (-p)
quit - exit ssh subsystem
user - username on remote host
Note that this program can only bind local ports in the range
8000-8999, inclusive.

Those of you who have used the cr-48’s ssh client before probably took note of the addition of a ‘key’ option. This new feature means that users can finally use their cr-48 to connect to a computer via ssh using a key rather than a password. I have not found much documentation on using this new feature, so I figured I would document how I got it working.

First, I copied my ~/.ssh directory from my primary laptop (not my cr-48) to a web-accessible location. Then, from my cr-48, I proceeded to navigate to the key I wanted to use and download it (just use the default location). Next, I launched a terminal (Ctrl+Alt+T).

crosh> ssh
ssh> user nhandler
ssh> host somehost.com
ssh> key /home/chronos/user/Downloads/mykey
ssh> connect

Assuming you followed these directions, you should get prompted for the password for the ssh key. Upon entering it, you will have an active ssh session to somehost.com.

This should help out many users who do not like to allow password logins for ssh.

Edit: I just realized that it is not necessary to enter the full path to the key. If you enter an incorrect path (such as trying to use ~ instead of /home/chronos), the following message is displayed:

File '~/user/Downloads/mykey' is not a valid key file. Key files must reside
under /media or /home/chronos/user. Key files in the Downloads directory may
be specified with an unqualified name.

As you can see, if the key is in /home/chronos/user/Downloads, you can drop the path and simply enter the key name.

Ubuntu IRC Council Nominations

As was previously announced on the mailing list, the terms of Benjamin Rubin and Jussi Schultink on the Ubuntu IRC Council are coming to an end. There is currently an open call for nominations to fill these spots. All nominations must be submitted no later than 23:59:59 UTC on Sunday, November 7, 2010 to irc-council@lists.ubuntu.com (private mailing list). The requirements for serving on the council are as follows:

• Ubuntu membership (Essential)
• Be able to demonstrate a competent technical knowledge of IRC.
• Be able to demonstrate a passion and desire for IRC and maintaining effective governance of our IRC community.
• Be familiar of the expectations listed in this charter and commit to executing them as part of the IRC Council

If elected, you will have the opportunity to serve on the council with a group of great people. Currently, the following people make up the Ubuntu IRC Council: Benjamin Rubin (term ending), Juha Siltala, Jussi Schultink (term ending), Nathan Handler, and Terence Simpson. As a member of the council, you would be responsible for several important duties in the IRC community including:

• Appointing or recalling IRC operators or determining criteria by which they are appointed.
• Resolving disputes between IRC operators
• With advice, feedback, and help from the IRC operators, maintaining the IRC Guidelines and associated infrastructure.
• Community Bug Reporting – the IRC Council should encourage the community to file issues using the Community Bug Process at https://wiki.ubuntu.com/ReportingCommunityProblems – this provides a good snapshot of where people feel there are problems in the community.
• Regular IRC Meetings – at least one public real-time IRC meeting per month with a public request for agenda items. They should have an Agenda wiki page to handle this process. The community can add agenda items and they are discussed and decided upon.
• Reporting – meeting notes and outcomes should be published using the TeamReports framework at https://wiki.ubuntu.com/TeamReports

The IRC Council has a lot of work to get done during the next 2 years, so if you are up for the challenge and meet the requirements, we would love to have you join the council.

Remember, there is only slightly over 1 week remaining in the nomination stage, so if you are interested, please do not hesitate in sending in your nomination.

If you have any questions about the nomination process or about serving on the IRC Council, please feel free to contact either myself or another member of the council; we would be glad to answer any questions you might have.

Ask Ubuntu Stats on Ubuntu Hall of Fame

Ask Ubuntu has been gaining more and more popularity each day. I recently created a script that parses http://stackexchange.com/leagues/31/week/askubuntu to find the top 10 ranked people for the week. Thanks to some help from the awesome Daniel Holbach, the output of this script is now displayed on the Ubuntu Hall of Fame. Keep up the great work everyone!