It started a month ago …
On July 24th, I was having coffee with a friend.
We talked a bit about employment opportunities in Morocco, challenges, perspectives … but mostly, problems.
One problem that came up was the huge fragmentation in the job market. More accurately, the fragmentation in the job offers/listings/postings. Then he said :
I wish I had a system that notifies me whenever a job opening that concerns me is posted, anywhere.
I was amazed by how simple the phrase sounded, but also, but how big the problem was.
At the moment, I was working on approximately 3 different projects and running almost 10 experiments.
I dropped everything.
Start. Think later.
Later that night, I went home, got a sketchpad and a pen, fired up my laptop, closed all the windows/tabs, and started working on what later became JobFinder.ma
At first, it seemed trivial: make a few crawlers and robots, invoke a few web-services, hook everything to a database, run everything. Smooth.
As the hours went by, the challenge got bigger and bigger. Making a snappy search engine e-no-easy task.
I came up with 4 major moroccan job portals to crawl (for the beta stage), then the hard (very hard) part came along: Three out the four portals had really bad HTML. The parsers I used to work with couldn’t even figure out if the document was HTML. I had to make a new parser in Ruby (which appeared to be super-slow when it came to dealing with, well, bad data; so it got ported to C++), and a machine learning algorithm to figure out the semantics of each job posting, … (let’s not get bored with the details for now).
At the end of a 48Hours hackaton, a basic prototype (single-crawler + database + web-service + Ruby/JS front-end) were up and running. YES !
It will have probably have tens of thousands of offers.
30 days later, I went to prod. Fired the initial seed robots … and launched as a public beta.
I realised the huge gap when it comes to dealing with 1.000s of records on a development server vs dealing with 1.000.000s at production scale; I faced unimaginable challenges, say, having 500+ concurrents requests to the database, not using Memcached (duh! thought it would launch quietly …), … Stuff that only someone with less experience backend-side would face.
At the end
To conclude this first JobFinder post, the main goal was to help unemployed people find a job, and help the other find better careers. Mainly to help a cause.
The second day of its launch, the website had 13.000+ indexed offers, 4100+ visitors, 16.000+ pageviews, and an average visitor duration of 3min14. And 450+ Facebook fans. Not bad for a start.
I’d like to finally thank everyone that was involved in the Alpha stage, either helping with the product development, giving feedback, or the Awesome guys who put ads for free on their apps/pages/websites !