Saher El-Neklawy

DAGs - Pratical Applications

2020-02-06T00:00:00+00:00

A presentation I have on what are Directed Acyclic Graphs and their practical applications during the EgyptJS meetup.

Checkout the slides on Slideshare

A Tale of Two Kitties

2019-03-07T00:00:00+00:00

The story of two office cats and how we optimized their feeding schedule.

Read the full post on Medium

Living on the Edge (Node)

2019-01-07T00:00:00+00:00

A tale of how using a Content Delivery Network is essential from day one for dynamic content.

Read the full post on Medium

Semi-Automated Text Cleaning in R

2015-12-26T00:00:00+00:00

Cleaning real life textual data is hard. Weather it’s convention inconsistencies, manual data entry mistakes, or a myriad of other reasons, reaching a consistent representation is essential. This is most pronounced when dealing with categorical variables (like city or job title) where an inconsistent representation would give rise to superfluous categories that do not exist in reality.

Read the full post on Medium

Writing your own dplyr functions

2015-12-18T00:00:00+00:00

dplyr is awesome, like really awesome. The thing I like most about it is how readable it makes data processing code look. But how to write your own functions that make use of dpyrl convesions?

Read the full post on Medium

Intro to Server Monitoring

2015-09-05T00:00:00+00:00

On 29/8/2015 I presented an introductory session about server monitoring. This talk is part of the Cairo DevOps Meetup.

The content of this talk can be found in this repo.

Intro to Spark Meetup Talk

2015-08-29T00:00:00+00:00

On 29/8/2015 I presented an introductory session to Apache Spark. This talk is part of the Cairo Data Science Meetup.

The content of this talk can be found in this repo.

Adding local latex packages to texlive in ubuntu

2011-02-19T12:52:19+00:00

When having custom packages to use for a document, you need to put them next your latex files when compiling. This causes problems when you need these packages for many documents. The solution is to add these packages to the latex installation. This can be done for the current user by creating a latex tree in the $HOME/texmf directory. The structure of this directory needs to follow the texmf-local described here. This directory is searched by default by the texlive installation.

Portable Meld on Windows

2011-02-12T13:51:14+00:00

Often I need to compare and merge files or folders together for changes between them. Gladly, many applications exist for such a task. This can come as using the diff command line tool, sadly it is not visually pleasant. Therefore several visual interfaces to ease the comparison side by side. Examples of such are:

A detailed comparison between the them and others can be found here. My personal favorite of these is meld (specially for folder comparison). On a Linux system, it is quite easy to set up, specially if it is deb bases because there is a package for it. A problem shows up when wanting to use meld in Windows. After some googling I found this guide showing how to install it for windows. Using this guide, I managed to get it to work, but I prefer a portable installation to take it anywhere.

Following are the steps I followed to get a portable version of meld up and running. Take care that performing the steps need administrative rights, but it can run anywhere.

Create a directory to contain meld binaries and its dependencies. Let us reference to it as <meld install dir>
Download portable python 2.6, and install it under <meld install dir>\Python26
Add the portable python installation to the registry. This is done by opening the portable python interpretor an running the script from here
Download the GKT+ 2.2 run time all in one bundle from here. Extract it to <meld install dir>\gtk+
Download an install the latest PyGTK libraries. These are the PyGTK, PyCairo, and PyGObject modules. This will install to the correct python based on the registry entry from step 3.
Download meld 1.5 from here. Extract it to <meld install dir>\meld-src
Create a file "<meld install dir>\start-meld.bat". The content of this file is:
[sourcecode language="bash"]
@echo off
set PATH=%CD%\Python26\App;%CD%\gtk+\bin
python meld-src\bin\meld
[/sourcecode]

And that is it, run start-meld.bat and enjoy :)

A DRY KISS

2010-10-29T14:13:53+00:00

Absolutely Don't Repeat Yourself and Keep It Simple, Stupid! (A DRY KISS) .

DRY and KISS are common concepts between software developers, but are very difficult to follow. Specially when you start with some idea, run with it, and it starts to layer up in complexity. Without even recognizing it goes out of control. With each of there added layers, you might find yourself repeating things without even recognizing the fact.

Drawing the very fine line between simplicity and and complexity is a very difficult task. It gets even more difficult without taking a step back to look at the big picture, the original aim, the purpose.

This is why an external review of the matter by a 3rd party is often vital. It gives perspective that helps the general good. Such a review process is often excruciating, painful, and long. But in the end of day, without it there is no real gain.

A a final note, I believe that A DRY KISS is something to try and follow in general.

Proposal: Hierarchical Unit Testing for less woes

2010-10-02T15:54:29+00:00

Disclaimer: The following content is of an idea which might be already existing, so please feel free to comment if you find it in existing testing frameworks :)

A common problem I face during test-driven development is when having to tests (for ease i'll call them T1 and T2). T1 and T2 test the functionality of F1 and F2 respectively. The fun starts when F2 is dependent on F1, and both tests fail. In a test suit of thousands of tests, it would be hard to see the correlation from only the report of running the test suite.

It gets worse when the dependency between tests is not direct. For example if F3->F2->F1 (-> means "depends on"), where only T1 and T3 fail and T2 is fine. These relations need a very careful reviewer of the test results, and one who is well aware of the relations between components and their tests. This might be impossible in HUGE teams spanning many components.

But what if the testing framework was aware of these relations, and was able to report them. Such relations could be represented for example as new annotations in JUnit. With building the relation graph between tests it would be easy show that T2 might have failed because T1 failed.

So this is an open call, any one willing to try creating this, if it is not already out there?

Google instant search: A better spell checker and calculator

2010-09-16T12:06:30+00:00

One of my main uses for Google search is not the searching, but has slowing move towards often queries just for the sake of spell checking. Recently Google released its instant search. With that, I can easily see my spelling mistakes as I type, no more need to press enter :).

What I found by accident and found it even more useful is using instant search as a calculator. Some of you might know the Google easter egg about the answer to life universe and everything. Google calculator has been part of Google's search features for some time now. With the instant search, the results of the calculator is also changed live. A cool and useful example (that was the way that made me discover this) is the binomial coefficient function. Give it a try and play around with it here.

Happy lazy calculating :)

Layar POI service using Google App Engine

2010-04-04T16:54:51+00:00

Layar is a cool handheld augmented reality application that allows to overlay "Layers" on the image seen by a handheld device's camera. One can think of these layers as content that is seen based on your current location. This allows to overlay digital data over actual live imagery.

The Layar API depends that the source of data (Points Of Interest) is a RESTful web service, that sends an HTTP GET request, and expects back a JSON object. The details of the GET parameters, and the required JSON object could be found here in the layar API documentation.

To create a layer, one needs to provide such a service that provides the Points Of Interest (POI). Such a service could easily be written and provided by google app engine. This article will discuss how to do so using the google app engine python SDK, and is highly depends on the getting started guide for google app engine through python. Sections 1-4,6 are sufficient for understanding of the coming content.

Handling Requests

Since requests from layar to the POI web service come in the form of GET requests, the parameters can simply be accessed in the request handling method through:
[sourcecode language="python"]
self.request.get(parameter_name)
[/sourcecode]

POI Response

The POI response that layar expects is a json object. The API place many restrictions on the response. The first comes in the content type of the HTTP response. Thus, it has to be set as follows:
[sourcecode language="python"]
self.response.headers['Content-type'] = 'text/javascript; charset=utf-8'
[/sourcecode]

Another restriction is in JSON object returned as a response, where almost all the fields are required by layar. To make things simpler, the object is to be represented as a python object. This could be later converted to a JSON object. The JSON object responded with could be represented as follows (as a minumum):
[sourcecode lanaguage="javascript"]
{'layer':'layer name', 'hotspots':list_of_POI, 'errorCode':0, 'errorString':'ok'}
[/sourcecode]
Where hotspots is a list of POI object. These object could be represented as:
[sourcecode language="python"]
class POI:
def __init__(self,poi_id,title,lat,lang):
self.actions = []
self.id = poi_id
self.imageURL = None
self.lat = lat
self.lon = lang
self.distance = None
self.title = title
self.line2 = None
self.line3 = None
self.line4 = None
self.attribution = ""
self.type = 0
self.dimenion = 1
self.transform = {'rel':True, 'angle':0, 'scale':1.0}
self.object = {'baseURL': ""}
[/sourcecode]

From python to JSON

As python follows a batteries all included strategy, there are libraries that convert python dictionaries to JSON objects. Even though google app engine uses python, the "json" module is not available. Thankfully, an equivalent one is found through "from django.utils import simplejson". It could be used to convert a python dictionary (that looks frightfully like a JSON object) to a JSON object string as follows:
[sourcecode language="python"]
simplejson.dumps({'layer':'guc', 'hotspots':poi_list, 'errorCode':0, 'errorString':'ok'})
[/sourcecode]
The problem with simplejson is that it only takes dictionaries or lists. Thus, this poses a problem when using the POI class mentioned earlier. Again, python comes with a rescue, where a dictionary representing the object could be obtained as follows:
[sourcecode language="python"]
poi = POI('C1','C1',29986707,31438864)
poiDictionary = poi1.__dict__
[/sourcecode]

Putting the code together

Now that we have the POI object, and the conversion mechanism, we are ready to have a request handler for layar requests. An example of static POIs is as follows:
[sourcecode language="python"]
from django.utils import simplejson
from poi import POI
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app

class POIHandler(webapp.RequestHandler):
def get(self):
self.response.headers['Content-type'] = 'text/javascript; charset=utf-8'
#latitude and longitude is an integer will be divided by 10^6
# so take care of accuracy after the division
poi1 = POI('C1','C1',29986707,31438864).__dict__
poi2 = POI('C2','C2',29986744,31439272).__dict__
poi3 = POI('C3','C3',29986995,31438923).__dict__
poi4 = POI('C4','C4',29987153,31439245).__dict__
poi5 = POI('C5','C5',29986326,31438810).__dict__
poi6 = POI('C6','C6',29986688,31438569).__dict__
poi7 = POI('C7','C7',29986442,31438370).__dict__
pois = [poi1,poi2,poi3,poi4,poi5,poi6,poi7]
# final getPOI response dictionary
d = {'layer':'guc', 'hotspots':pois, 'errorCode':0, 'errorString':'ok'}
self.response.out.write(simplejson.dumps(d))

application = webapp.WSGIApplication(
[('/',POIHandler),
('/getPOI',POIHandler)],
debug=True)

def main():
run_wsgi_app(application)

if __name__=='__main__':
main()
[/sourcecode]

Updates

With the new layar 4 api coming soon, and the new features being in beta2, un update of how to add one of my favorite new features is needed. This is having different actions on the entire layar. It can be easily done by adding an action attribute to the final getPOI response dictionary (see code above). An example of the addition is as follows:
[sourcecode language="python"]
d['actions'] = [{'uri':'http://google.com','label':'action on the entire layar'}]
[/sourcecode]

SVN post-commit hook cronjob

2010-04-04T12:08:17+00:00

Few SVN hosting sites provide ssh access to their svn servers. Such is needed to create hooks, like a post-commit hook that sends an email with every commit. Such hooks are often required by the development team. This article discusses the creation of a cron job that polls the SVN repository to simulate a post-commit hook that sends an email with every commit.

Setting up sendEmail

sendEmail is a nice tool to send emails via command line. This is needed to script in the cronjob sending an email. On ubuntu, install it from apt-get by:

sudo apt-get install sendemail libio-socket-ssl-perl libcrypt-ssleay-perl

The latter 2 packages are needed for connecting to an SMTP server that uses a secure connection (like Gmail, which is used in the example).

The post-commit hook script

The following is a python script, that could be added as an entry in the cron table
[sourcecode language="python"]
#! /usr/bin/env python

import os,re

log_file = LOG_DIR
svn_log_seperator = '------------------------------------------------------------------------'
svn_dir = SVN_DIR
mail_filter = '[special mail identifier in subject]'
message_file = '/tmp/commit_mail'

mail_from = MAIL_FROM
mail_to = MAIL_TO

smtp_server = "smtp.gmail.com:587"
smtp_user = SMTP_USER
smtp_pass = SMTP_PASS

def send_mail(subject,mail):
mail_file = open(message_file,'w')
mail_file.write(mail)
mail_file.close()
mail_command = 'sendEmail -f %s -t %s -u "%s" -s %s -xu "%s" -xp "%s" -o message-file="%s"' % (mail_from, mail_to, subject, smtp_server, smtp_user, smtp_pass,message_file)
print mail_command
os.system(mail_command)

print "analyzing old log file"

old_log_f = open(log_file)
old_log = old_log_f.read().split(svn_log_seperator)
old_log_f.close()

print "updating svn"
os.system("/usr/local/bin/svn update %s" % svn_dir)
os.system("/usr/local/bin/svn log %s > %s" % (svn_dir,log_file))
print "got new log"

new_log_f = open(log_file)
new_log = new_log_f.read().split(svn_log_seperator)
new_log_f.close()

delta = set(new_log) - set(old_log)

for commit in delta:
l = [x for x in commit.split('\n') if x]
details = l[0].split('|')
subject = "%s %s %s" %(mail_filter,details[0],details[1])
f_diff=os.popen('/usr/local/bin/svn diff -c %s %s' % (details[0], svn_dir))
diff = f_diff.read()
f_diff.close()
mail = "%s\n%s" % (commit, diff)
send_mail(subject,mail)
[/sourcecode]

Perquisites before running the script

Having a working copy of the svn reposity

svn log > $PATH_TO_LOG_FILE

Developer vs. User <=> Driver vs. Mechanic

2010-04-01T17:51:50+00:00

Hypothesis: The Developer User relation is equivalent to the car driver mechanic relation.

Proof:
Through my few numbered years of driving in the lovely streets of Cairo, i faced car troubles every now and then. In the case of such circumstances, I head to my trusty mechanic. From here, starts the problem, when I am driving my car, and i notice a problem, it is very difficult to reproduce by the mechanic. Such a reproduction often happens when it's too late.

Here, comes the fitting analogy, mechanics are car using experts. They manage to use cars in an expert manner, one that hides any possible problems in the car.

The same maps to the computer software users, where they see problems that developers are unable to reproduce, simply because of their computer usage habits that are so much different from that of end users.

Quod Erat Demonstrandum (QED).

Thus, the lesson to learn here is, every time you want to feel like a user, think of yourself driving your car, and weird things starting to happen.

Life: A python perspective

2010-03-17T01:40:32+00:00

[sourcecode]
from people import friends

while not dead:
try:
something_new()
except LifeIsToughError:
pass
[/sourcecode]

Breaking up with your code

2010-03-04T20:14:30+00:00

A famous quote from The Cathedral and the Bazaar is "Plan to throw one away; you will, anyhow". This never hits a programmer with it's true meaning until they have to throw aways a large bit of code base that they invested time and effort in. Coders tend to build an emotional connection towards every letter of code they wrote (I know i sure do). On the other hand, needs arise where obstacles render current software "unfixable". This confronts with the painful fact that change must be done. Taking an objective stand point in that decision is a tough decision, but facts are fact in face of improvement. In the end of the day, breaking up is tough but a fresh start would most likely end up in more bulletproof code.

The Recurring Beauty

2009-12-27T17:38:57+00:00

Recently, I stared reading one of the essential books for a computer scientists, namely Concrete Mathematics. After the first chapter, it really got me thinking "Oh My!! Recurrences could solve anything." This is of course not matched to the thrill of the challenge in funding recurring patterns in a structure.

Along my college years, colleges tend to step away from recurrences. In fact, many cases it is the easier approach to tackle. Furthermore, it could be in some cases the more efficient approach. This reminds me of many algorithm techniques that largely depend on finding substructures or small solutions that are used for generalizations. The most vivid example comes to me in the form of memoization in dynamic programming (DP). For me the corner stone of DP is finding the "magical" recurrence. Thus, from super slow, to super fast (lots of memory use though :D).

Other examples of how recursive thinking makes complex problems elegantly simple could be seen in graph theory. Just by examining some properties on nodes, arcs, degrees, huge complexities just boil down to some elegantly beautiful properties that are easily generalizable.

Such beauty could be easily seen when recurrences could end up in something that is visually breath taking. An example are Fractals, a stunning example of how simple math with recursion lead to amazing imagery.

Two Columned LaTeX Presentation Slides

2009-12-07T12:01:36+00:00

Coming from a Power Point background of creating presentation slides, i am often tempted to used a two column approach when laying out the content of my slides. This is where text is shown on the left column and related pictures are placed on the right.

After i decided to to move all my document creation work to be in LaTeX, I started to also create my presentations that way too. Such a feat is made possible using the beamer document class, whose user guide could be found here.

The environment for a single slide is the "frame" environment. It is used a follows:

[sourcecode]
\begin{frame}{frame title}

\end{frame}
[/sourcecode]

To be able to split the frame (slide) into two columns, the "columns" environment is places inside the slide content. It is used as follows:

[sourcecode]
\begin{columns}
\column{column_width} %first column
column 1 content
\column{column_width} %second column
column 2 content
\column{column_width} %third column
column 3 content
...
[/sourcecode]

It is best to have the column_width in terms of the text width. So for example:

[sourcecode light="true"]
\column{0.5\textwidth} %half slide's text width
[/sourcecode]

An example for a two columned slide looks as like:

[sourcecode collapse="true" wraplines="true"]
\documentclass{beamer}
\mode<presentation>
{
\usetheme[width=2cm]{Hannover}
\setbeamercovered{transparent}
}
\title[Presentation]{Presentation}
\author[Saher Mohamed El-Neklawy]{Saher Mohamed El-Neklawy}

\begin{document}
\begin{frame}{Frame Title}
\begin{columns}
\column{0.5\textwidth}
\begin{itemize}
\item Item 1
\item Item 2
\item Item 3
\end{itemize}
\column{0.5\textwidth}
\includegraphics[width=\textwidth]{i1.eps}

\includegraphics[width=\textwidth]{i2.eps}
\end{columns}
\end{frame}
\end{document}
[/sourcecode]

[caption id="attachment_66" align="aligncenter" width="300" caption="Two Columned Slide"][/caption]

The next step is to sequence the appearance of the images along with the images. In the beamer class, this could be done using the "" operator, following a certain component in the frame. The "n" represents when the component appears in the frame. For example:

[sourcecode]
\begin{itemize}
\item<2> Item 1 % seen after 1 transition
\item<3> Item 2 % seen after 2 transition
\item<4> Item 3 % seen after 3 transition
\end{itemize}
[/sourcecode]

In a sense, the "" splits a frame into n slides, placing the respective component in the nth slide of that frame. For more advanced uses of "", check out the beamer class user guide.

To do the same for images, the "\visible" or "\only" macros could be used along with the "" operator.

With using "\visible", the slides would now look like:

[sourcecode collapse="true"]
\documentclass{beamer}
\mode<presentation>
{
\usetheme[width=2cm]{Hannover}
\setbeamercovered{transparent}
}
\title[Presentation]{Presentation}
\author[Saher Mohamed El-Neklawy]{Saher Mohamed El-Neklawy}

\begin{document}
\begin{frame}{Frame Title}
\begin{columns}
\column{0.5\textwidth}
\begin{itemize}
\item<2> Item 1
\item<3> Item 2
\item<4> Item 3
\end{itemize}
\column{0.5\textwidth}
\visible<2>
{
\includegraphics[width=\textwidth]{i1.eps}
}
\visible<3>{
\includegraphics[width=\textwidth]{i2.eps}
}
\end{columns}
\end{frame}
\end{document}
[/sourcecode]

[caption id="attachment_74" align="aligncenter" width="300" caption="Sequenced Two Columned 1"][/caption]

[caption id="attachment_75" align="aligncenter" width="300" caption="Sequenced Two Columned 2"][/caption]

The problem with "\visible" is that it keeps the place of the component from the previous transition reserved in the next, just not seen. The fix to this issue is the use of "\only", where the space for components is not reserved across transitions. Such makes the slides look as follows:

\begin{document}
\begin{frame}{Frame Title}
\begin{columns}
\column{0.5\textwidth}
\begin{itemize}
\item<2> Item 1
\item<3> Item 2
\item<4> Item 3
\end{itemize}
\column{0.5\textwidth}
\only<2>
{
\includegraphics[width=\textwidth]{i1.eps}
}
\only<3>{
\includegraphics[width=\textwidth]{i2.eps}
}
\end{columns}
\end{frame}
\end{document}
[/sourcecode]

[caption id="attachment_76" align="aligncenter" width="300" caption="Two column slide using \only macro 1"][/caption]

[caption id="attachment_79" align="aligncenter" width="300" caption="Two column slide using \only macro 2"][/caption]

When taking a closer look at the output pdf, it will be observed that there is a slight change in the position of the bulleted list. This is most seen between pages 3 and 4 of the pdf. Such is due to the nature of the \only, as it does not reserver the position of component across transitions of the frame. Thus, the location of the bulleted list differs when there is a transition with a slide having a slide to one without, and vice versa.

This problem could be fixed by going back to the "\visible" macro, as it reserves the spaces for components. The difference this time is is that the location of the image in the slide will be forced. This could be done using the "picture" environment as follows:

[sourcecode]
\begin{picture}(0,0)(x,y)
\put(0,0){\includegraphics[width=\textwidth]{i1.eps}}
\end{picture}
[/sourcecode]

Take care to consider the x and y values from the top right corner of the column.

The final code results in this pdf, and looks like follows:

[sourcecode]
\documentclass{beamer}
\mode<presentation>
{
\usetheme[width=2cm]{Hannover}
\setbeamercovered{transparent}
}
\title[Presentation]{Presentation}
\author[Saher Mohamed El-Neklawy]{Saher Mohamed El-Neklawy}

\begin{document}
\begin{frame}{Frame Title}
\begin{columns}
\column{0.5\textwidth}
\begin{itemize}
\item<2> Item 1
\item<3> Item 2
\item<4> Item 3
\end{itemize}
\column{0.5\textwidth}
\visible<2>
{
\begin{picture}(0,0)(40,50)
\put(0,0){\includegraphics[width=\textwidth]{i1.eps}}
\end{picture}
}
\visible<3>
{
\begin{picture}(0,0)(50,50)
\put(0,0){\includegraphics[width=\textwidth]{i2.eps}}
\end{picture}
}
\end{columns}
\end{frame}
\end{document}
[/sourcecode]

Open source Projects, The Life Cycle

2009-11-21T20:08:01+00:00

Recently, I have been involved into the development of an open source project, that is currently under major modifications. Seeing this change happening before my eyes got me to think, how do open source projects evolve, and how do exiting well matured (old) projects reached their current state.

I feel that most open source projects start with a crazy idea, usually driven by some sort of need (the itch as called by The Cathedral and the Bazaar). But when one thinks about it, not many people have the guts to go out in the open to satisfy that drive. Mostly people who do aim to get an up and running prototype that shows the idea to the community. This is mainly to get people in the community excited, and people who had a similar itch involved.

To reach that initial prototype, the team working is often small, as the community knows nothing yet. The lack of the outsider perspective might limit the developers to their own coding style, and the drive of getting a prototype out might loose focus on certain coding aspects.

After the prototype is out, the effects of the bazaar model is seen, where bugs are fixed, code is optimized, improved, and documented. At this stage, the project takes it's first step to maturity.

The changes over time could range the minor to the drastic. The main issue that developers wanting to get involved with an evolving project, whether rapidly or slowly, patience is need. This is due to the need to take time to obtain a deep understanding of the code. Furthermore, patience is needed when a project is changing rapidly, as they mostly could end up in code breaks, but keep in mind they will eventually be fixed (possibly by you). Such an aspect also requires high flexibility.

After these many iterations by the community, the project starts to stabilize into a state of maturity. Usually this is called the 1.0 release. After that stage, progress starts to slow down, and focus starts to go to having new features. The main code base is rarely changed unless with major following released, or core feature additions that require that due to the structure of the code.

Running launchpad remotely

2009-11-04T20:58:28+00:00

Launchpad is a software collaboration platform developed by Canonical (famous for Ubuntu :D). Recently the the launchpad project went open source. This allowed people to host copies of it locally. The process for the installation process could be found here.

The next natural step is to configure remote computers to be able to access launchpad. After following the how-to by launchpad, and trying the access the local copy of launchpad from an external source, the following error is shown by the running launchpad server:

NotFound: Object: <canonical.launchpad.webapp.servers.ProtocolErrorPublication
object at 0x8157b50>, name: ''

The solution to this lies in the /etc/hosts file on the client accessing the lauchpad copy. The file should contain the following:

{ip for server}      launchpad.dev answers.launchpad.dev api.launchpad.dev
bazaar-internal.launchpad.dev beta.launchpad.dev
blueprints.launchpad.dev bugs.launchpad.dev code.launchpad.dev
feeds.launchpad.dev id.launchpad.dev keyserver.launchpad.dev
lists.launchpad.dev openid.launchpad.dev ppa.launchpad.dev
private-ppa.launchpad.dev shipit.edubuntu.dev shipit.kubuntu.dev
shipit.ubuntu.dev translations.launchpad.dev
xmlrpc-private.launchpad.dev xmlrpc.launchpad.dev

{ip for server}      bazaar.launchpad.dev

{ip of the server} is the ip of the machine containing the running copy of the launchpad server. Also take care the lines starting with {ip of the server} until an empty line should be a single line in the file. For client systems that are not linux, check out the Wikipedia article concerning hosts for the files equivalent to "/etc/hosts".

To run launchpad, access https://launchpad.dev/.

On a side note, when installing launchpad, it does not add scripts for running automatically at startup, this process has to be done manually.

Searching Source Code: LXR and grep

2009-11-01T23:07:02+00:00

Looking for a certain functionality in someone else's code could prove to be a tough challenge. This is increased by the size of the given project. Many systems exist today facilitating easy search in a project. For Java, eclipse is best used for that task, as it contains a strong in java class indexing, classification, and searching.

Some projects provide for you a portal to search their code base. Famous examples of these are LXR for the Linux kernel, and it's customized version MXR used for Mozilla projects.

When all the above options are not available, the alternative exists natively in most linux systems. This is namely "grep". Grep is a strong command for regular expression matching. On if it's features is that it can also match to the content within files. Putting that in mind, the following command could be very useful:

grep -R "search query" .

This looks recursively for all mentions of search query in all files under the current directory. This prints output in the console the file name where this query was found, and the line that included it.

To be able to store this search result in a file, the output could be piped as follows:

grep -R "search query" . > log.txt

"Bug Free Zone"

2009-11-01T22:42:17+00:00

As you can see from the title of my post, it's all about bug. But not bugs as in insects, but as in computer bugs/problems. In the open source development course, we were introduced to the mozilla development ecosystem.

This introduction was done via Thunderbird, the Mozilla email client. The first step was to compile a debug build for Thunderbird from source. Instructions for that could be found here. Along the process, several bumps along the way were met, but managed to get by them.

After the successful compilation, I had to start tackling the the bug famous in the open source educational community. The Thunderbird auto linkification bug. This bug is usually exists in Thunderbird, and usually mainly found under the test bugzilla.

To start on fixing that bug, a search of the source code is needed. Since Thunderbird is a Mozilla project, they offer an on-line portal for searching their code using MXR. This tool searches the content of the repository. After some searching, the best query to find the file related to this problem was "mailto". This is because the bug is related to the mail rendering component, when looking at it from purely a user's point of view.

At finding the correct file, the hacking started. Once that was done, I generated a patch of the file i changed from inside the comm-central/mozilla folder.

[caption id="attachment_41" align="aligncenter" width="288" caption="Thunderbird Bug"][/caption]

Once the results of my fixing looked promising, i decided to submit a bug report on the test bugzilla containing a patch. But before the submission, i decided to test my patch first on an automated reviewer. To my surprise, it showed me problems that would have never came up on my mind.

[caption id="attachment_42" align="aligncenter" width="450" caption="JST Review"][/caption]

Passing the automated reviewer gives one a boost of confidence, even if it responds to you with the message of "Congratulations! I found no coding problems in your patch. That does not mean that it is ready for check in though... you still need to pass something more intelligent than a script. :-)".

Now, it was time to submit the bug, attach the patch, and request review. This was amazingly simple. Just filling a form, being very detailed, and attaching the patch file. After the submission of the bug report, I requested reviews from 2 parties, which I am still waiting for a response for.

On a final note, I found this exercise and experience very entertaining and thrilling. The rush of hacking the code, and submitting was a new feeling for me. This test experience drove me to submit an actual bug related to bespin's new refactoring code. Hopefully I get a good review on it too, and possibly a commit :D.

Converting System::String to char *

2009-10-31T16:05:14+00:00

The official way to convert a visual c++ System::String to a char * could be found here. Such strings are everywhere in windows forms applications.

A simpler way (for people who are used to using standard c\c++) to do the conversion is as follows:
void string2charPtr(String ^orig, char *&out) { int length = orig->Length; out = new char[length+1]; for(int i=0;i<length;i++) out[i] = (char) orig[i]; out[length] = ''; }

Please feel free to comment on the function above.

Opensource Needs Open Internet

2009-10-26T10:24:30+00:00

When installing opensource software, dependencies are are usually downloaded over the internet. This is done via build automation scripts. Putting this in mind, when trying to install software in a location with restricted internet access, downloading dependencies is not possible. Thus, installing software is challenging.

This was seen while trying to install bespin locally. Under restricted access, was not possible. With open internet, too 3 minutes.

Processing.js:Reinventing the wheel

2009-10-26T07:45:51+00:00

HTML5 is the new standard for web pages. One of the most exciting features that it introduces is the canvas tag. This component allows rendering vector graphics on the browser. This was not previously possible with older HTML versions.

With that advancement in HTML, new JavaScript libraries emerge to be able to create vector visualizations at the client side. One of these libraries is Processing.js, which is a port of the java processing library.

For a first look at processing, the examples show very exciting examples. But the week point of the framework shows when try to develop an application that required much interaction week. To reach such an application, the "processing.js" written has to be highly optimized. No room for unnecessary code execution, use of floats or doubles, cache as much as you can, etc....

My other issue with Processing.js is the fact that they ported from the java library everything, even the syntax. This comes to be very frustrating to web developers. The first reason for that is most web developers are used to the syntax of JavaScript, so having things like int x=0; and int [] arr = new int[4]; come as a big surprise. This could be good for the java developer, but not the the web developer.

My other bone to pick with Processing.js is how primitive it is relative to the current state of the web. There are no components, and you can't define events on specific components. Everything has to be done from scratch. Further strengthening the point of needing highly optimized code.

Backing to the title of my post, Processing.js tries to provide an alternative for Flash. In the web, the major disadvantage of Flash is it's isolation from the rest of the webpage, making it an independent entity. Processing.js does not over come this problem, as the visualization code for each canvas is a world of it's own. It can only access normal JavaScript code in the same page (which is possible with ActionScript 3's ExternalInterface). This problem even escalates with lack of direct communication between different canvases directly using Processing.js code. Furthermore, Flash is currently cross browser compatible, unlike the canvas tag due to it's new addition to the HTML standard.

Putting all the above factors in mind, I feel that using a the much mature flash plugin libraries which are closer to web developers is the best option currently in the market until the canvas tag matures more with richer libraries.

The ubiquitous jQuery, the nouns, and the documentation

2009-10-19T19:19:10+00:00

With starting up the German University in Cairo Open Source Community (g-osc) working along the open source course, we were assigned to write ubiquity commands for our community to use. This would make it easier for users to access different functionalities. I choose to build a command that would submit issues to the g-osc issue tracker.

As I had some experience building very simple commands for some of my own day to day internet surfing, I decided to focus on this command a bit more to see the full potential of ubiquity. This was to provide the means of submitting an issue in the most natural way. As I went through details of the ubiquity tutorial, I figured that a natural approach for defining how the command will be used is as follow:

gosc post issue {issue title} to {user issue will be assigned to} as {type of issue}

Such a formulation is similar to how it would be written in normal English.

The first design question was, how will be the issue submitter be known. This is automatically figured out by mediawiki, though the browser session. In other words, if a user is already logged in, then mediawiki knows this information on its own. With that issue out of the way, there was no need to write specific login functions.

The next step, was to reverse engineer the issue posting form, so that AJAX would be able to send the information as if a user was filling the form manually. After obtaining the form information, the next obstacal was actually send the data. This was straight forward task, due to the inclusion of the JavaScript jQuery library within ubiquity. This a very strong feature, due to strength of jQuery.

With that out of the way, the command was able to submit issues to the tracker. The next step was to be able to format the command natually, and to provide auto-completion for the available users and types of issues. This was done using ubiquity nouns. These tell ubiquity that a certain argument has a specific format. This idea is same as auto-completion, where the specific format is a specific set of word. In both the users and issue types, this data has to be dynamically loaded. Again, jQuery was used to load the form page, and parse the data to obtain the data inside the combo boxes.

A problem that comes up with auto-completion and loading dynamic data, is the need for doing the AJAX call with every key stoke. This is very expensive. The first solution that came to my mind was to cache the data, and refresh it every time the ubiquity command line is loaded. Sadly, as i mentioned in a previous post, the ubiquity documentation is not very accurate. Thus this cache technique was not successful, as the function for handing the loading function was not called. The alternative solution, was to cache the content only once.

With both the auto-completion working well, and a preview of the possible users present the command was complete. The complete code for the issue submission command could be found here.

Finding your true computer name on the network

2009-10-13T18:57:35+00:00

In many cases, it is essential to find the name of a computer over a local network. Such fits well when on a DHCP network, and need to always reach a certain machine.

An easy way to obtain this data, is via the python socket library. The ip of the desired machine is obtained via ipconfig (windows) or ifconfig (linux). For the sake of the example, assume that the ip from the DHCP network is 10.0.0.5.

In the python interpretor, execute the following code:
import socket true_name = socket.gethostbyaddr("10.0.0.5")[0] print true_name

The code above prints the true name of the machine, as seen on the network. This name is not always the machine name. For example the machine name could be "saher-desktop", while the name on the network is "saher-desktop.local".

Software versions, documentation, and the PAIN

2009-10-12T19:52:12+00:00

When working on a piece of software using a framework or an API, the first source of help to rely on is the official documentation. As development on that code base continues, so does the documentation that changes accordingly. Sadly this is not always the case!!

At the time of this post, the latest stable release of the ubiquity project is version 0.54 and the version pulled from mercurial is 0.55pre8. I was trying to write an enhancement to provide feedback to a command's author. My testing environment had the latest stable version, and as the ubiquity wiki is under 0.5, I assumed it matches my currently running version. At the end of the wiki, a link is provided with the documentation of the CmdUtils library, containg the command CmdUtils.getCommand(id).

Such a command is very helpful as it returns the object representing the command matching the given id. Thus, details about the command's author could be obtained, and hence sending to them feedback. Being very excited about this, I gave it a test, and to my shock nothing worked. Even to my bigger surprise, firebug that is used for debuging ubiquity commands showed CmdUtils.getCommand is not a function.

This caught curiosity, so I opened the ubiquity extension's installation folder, and took a look at the "modules/cmdutils.js" file. With no surprise, the file did not contain the CmdUtils.getCommand function.

At the same time, Abdallah El Guindy was working on the same enhancement, reached the same conclusion, but had the function correctly working. When I asked about his running ubiquity version, it as the latest development version 0.55pre8. After taking and installing that version of the code, the "modules/cmdutils.js" file was the one matching the current online documentation.

Having the correct version number attached to the online documentation, would have saved development time and confusion. Such a problem is not restricted to the ubiquity project, but is also seen in other frameworks or APIs. The main lesson learned here, is that one has to be very careful during development, and not give up easily on finding the root of the problem.

On a small side note, this same problem with ubiquity 0.54 is also in 0.55pre7 (the current latest beta). So the CmdUtils.getCommand(id) is a brand new addition to the code base.

HCI Workshop

2009-10-08T15:01:14+00:00

My first experience being involved in a workshop was both exciting and educational. This aimed to introduce different technologies technologies to students of the course. They would then choose a technology to work with a project of their proposal.

The workshop started with my Flex presentation. I followed through the standard sequence of over viewing highlights of the framework and showing sample applications.

Following my presentation was the talk by Mohamed Azab regarding Silverlight and WPF. There was a strong competition between these technologies and Flex during his talk. Such competition is always healthy and gives a chance for people to explore different options to perform the similar development tasks.

The day continued with presentations regarding mobile development platforms, where interesting talks were presented by Bahia El Sharkawy (JAVA-ME) and Mariam Rady (Android Framework).

The workshop concluded with my talk on Multitouch surfaces. It contained an the architecture behind building a multitouch enabled software, and the idea for detecting multiple interaction points. Furthermore, a live demo of the Mini Touch & Write table was demonstrated.

In general, the feedback is very positive, and I hope the talks inspired the students for thinking of highly interactive application ideas.