My Experience at TechCrunch Disrupt Hackathon 2016

Last weekend, I participated in the TechCrunch Disrupt Hackathon in New York City. Here’s my demo.

Screenshot 2016-05-10 21.19.25.png

The story of how I got on that stage with that project is slightly more complicated.

I originally went to the hackathon as part of a team: me, Tom, Lowell, Shabnum, and Scott.

The hackathon took place at the Brooklyn Cruise Terminal, a very industrial-looking place.

We were one of the first teams to arrive, so we got to pick a good table.

Our project was in EdTech, and we called it Mindset. Scott has written about it here. My job was to set up and implement the Natural Language Processing backend server and its API endpoint. The application would send the server a syllabus, and the server would parse it into topics, tag each topic, extract dates and deadlines, and return a nice data structure with all of this information. Its topic extraction would be powered by IBM Watson’s Concept Insights API.

The hackathon began at around 1:30 PM on Saturday and the deadline to submit projects was 9:30 AM on Sunday. We worked on it without facing any real problems all through Saturday afternoon and into Saturday night.

The hackathon had tables for 89 teams total, plus a number of booths for sponsors.

Soon enough, it was past midnight. We were starting to get tired but we were fueled by three things: teamwork, our goal to complete the project, and caffeinated beverages.

IMG_1774.JPG

Still going strong at 2:15 AM.

Before we knew it, 4 AM rolled around. Some of my teammates went home to take naps or freshen up. Some found places to curl up. Regardless, we were driven by a singular purpose: submit our project before the 9:30 AM deadline and wow the judges at the 60-second demo.

Finally, at around 5:30 AM, we had completed what we’d set out to do! Our website was up and running, my NLP server was making calls to IBM Watson and interpreting the results correctly, and our backend server was fully functional and robust.

My team started prepping for the demo. I didn’t need to be involved, so I was left to my own devices. I was wide awake at this point, and I had around 4 hours to burn, so I decided to do some work on a project I’d been thinking about for a while.

I had been planning to make a Twitter bot that uses Agolo‘s API to summarize the contents of any URL you tweet at it. This would be a follow-up to a similar Slack bot that I created a few weeks ago. I thought to myself, what better time to get started on this project than 6 AM at the TechCrunch hackathon after having stayed up all night?

I got to work on it. I picked Python because I have some experience working with Tweepy, a Twitter library. I knew that I had to circumvent the 140-character limit somehow, so I had the idea to use images to display the summary. I used the Python Imaging Library (PIL) for that.

I set up the Twitter account, got my code running on my AWS server, and started testing it. I had to make a number of tweaks to the way I was using PIL in order to make the text look good enough to demo. PIL doesn’t automatically do word wrap, so I had to find a way to insert newlines into the text where it made sense.

Finally, with around 20 minutes left until the deadline, I hacked together a working Python script that could achieve my project’s goal!

I submitted it, deathly-tired, forgetting one important detail: submitting a project means I have to give a demo onstage. I was about to fall asleep, but this realization was a shot of adrenaline that kept me awake.

It was time for the demos to start.

IMG_1775.JPG

The auditorium from halfway back.

I sat in the audience and mentally prepared some things to say at my demo. I checked, double-checked, and triple checked that my project was working.

Meanwhile, my teammates Tom and Shabnum went up to present Mindset. They did a wonderful job despite the technical difficulties they faced. I was proud to see the end result of a long night of hard work being presented up onstage.

IMG_1776.JPG

Tom and Shabnum setting up the laptop for their demo.

They called up the next batch of presenters to wait backstage. There was a 10-minute break during this period, so I got to practice my speech a little and meet some of the other presenters.

Backstage at the control booth. The black wall with the green lights at the top is the stage’s backdrop.

Finally, I was next in line to go onstage, set up my laptop, and wait for the previous presenter to finish.

Photos taken just offstage. I was balancing my open laptop in one arm as I took these pictures. I probably should not have taken this risk.

Then, I presented. I don’t remember most of it. The lack of sleep, combined with the adrenaline, put me in a state where I was giving an impassioned presentation of my project instead of paying attention to the hundreds of faces looking at me from the audience.

I walked offstage and back to my seat. I came down from the rush, and my tiredness finally took over. It took a lot of effort to finish watching the rest of the presentations and the awards ceremony. Then, I finally stepped outside for the first time in many hours.

Finally, sunlight and fresh air. Well, as fresh as it gets in NYC.

It was sunny for the first time in a week. It was a strange feeling to finally feel the sun on my skin after many days, punctuated by an intense experience like that.

I somehow made my way home.

Then, I slept for 14 hours.

All in all, it was a really fun experience. It was like a marathon, but with my team to keep it light and make it enjoyable. My 6 AM decision to start working on my own project turned out well, but I wasn’t in my right mind when I made that choice. However, sleep-deprived-me chose to take a big risk instead of playing it safe, and that’s a lesson I can learn from him. My main takeaway is to challenge myself and push my boundaries whenever possible, because the reward is often underestimated and risk is often overestimated.

Advertisements

Setting up swap space on a Linux server

Recently, I set up an EC2 host. It was a smooth and relatively quick process.

However, the problem with the default image is that it doesn’t come with swap space. You can verify this by running:

swapon -s

I found this tutorial online that walks you through creating a swap file, setting it up, and using it as your swap space:

All About Linux Swap Space

That post doesn’t mention it, so I’ll say it here: you need to use sudo in front of each of those commands. Another thing missing from that tutorial is that as I was setting this up, the shell gave me this warning:

$ sudo swapon /swapfile
swapon: /swapfile: insecure permissions 0644, 0600 suggested.

So, I had to do this:

$ sudo chmod 0600 /swapfile

At the very end, you can check whether you were successful using the swapon -s command:

$ swapon -s
Filename Type Size Used Priority
/swapfile file 1048572 0 -1

How I Start

I recently found this website on Hacker News called “How I Start.” It’s a series of tutorials for learning new programming languages. Right now, there are only a few, but I think they’re expanding to add more.

I’m learning Go for one of my projects this semester, so their tutorial for Go is very useful:

How I Start. – Go (with Peter Bourgon)

He’s setting up a web server, querying a weather API, and displaying some results.

I’ll be setting up an identical toy webserver soon, and I’ll post my results here.

Bitcoin

It seems that Satoshi Nakamoto has been hacked or somehow compromised. This led me to learn more about Bitcoin and its history so far.

Like many others, I’m kicking myself for not buying a few Bitcoins back in 2009. I was a college student then, and I had no disposable income. Still, I was considering buying one or two, just to walk myself through the process and use an exchange or a client. Maybe it’s a good thing I didn’t get involved in it at all, since its legality and widespread acceptance are still big issues.

Anyway, I’m trying to drown out that regret by satisfying my fascination with the concept of cryptocurrency.

First, here’s the original whitepaper about Bitcoin:

Bitcoin: A Peer-to-Peer Electronic Cash System

This Bitcoin series on Khan Academy is a good introduction as well:

Bitcoin: What Is It?

This Wikipedia article has some interesting background:

History of Bitcoin

And finally, take a look at these photos taken at a Bitcoin farm:

Gallery: Inside a Top Bitcoin Mine in China

Character encoding

Every software developer needs to know the basics of character encoding. However, I find it a very dry and dull topic. So here are some entertaining introductions to it.

First, a video explaining Unicode, UTF-8, and its elegance.

Now, read this popular Joel on Software blog post:

The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

 

A demo of neural networks

I enjoyed this demo of how exactly neural nets work:

A visual proof that neural nets can compute any function

I liked the interactive demo. I feel like I have a better intuitive understanding of weight and bias mean.

I took a machine learning course during my undergrad, and it was focused more on formulas and implementing machine learning algorithms. I felt that it didn’t really give me a fully intuitive understanding of how these algorithms really worked. I managed to implement the algorithms in Matlab, and I was able to step through them and look at the result of each iteration, but it always seemed a bit too complex to grasp.

This article, with its interactive elements, walks you through how sigmoid functions work and what they actually do.

Learning Python – Plans for a meal picker

I have some basic experience with Python — which is to say I’m a beginner. I want to become more fluent in it. So, I plan to write a toy script using Python to help me decide what to eat for breakfast, lunch, and dinner.

I want to start it off as simple as possible. I want to model it after a lunchtime restaurant picker I wrote (in Perl) when I was working at Amazon. I manually pre-categorized a list of restaurants, placing each one in tier-1, tier-2, or tier-3. The script would then randomly pick one restaurant from each tier, and then choose one out of those 3 using a weighted random number.

Some simple improvements will come in the next iteration. I plan to perhaps incorporate what’s in my fridge and pantry. I’m not sure how to model that data and store it, though. It needs to be human-readable, so I’m thinking a text file. That’s how I stored my list of restaurants and their tiers. For my new script, I could write a frontend tool to manage this data, but that might be over-engineering this simple script.

Anyway, the purpose is just to get familiar with Python, so it doesn’t need to get too complex.

Learning LaTeX

One of my courses this semester, COMS 4241 – Numerical Algorithms and Complexity, will most likely require the use of LaTeX for assignments. So, I’ve been trying to learn it before classes start.

This is the tutorial I’m using: LaTeX Primer from tug.org

And here’s a useful web app that allows me to play around with LaTeX. It compiles and generates the document nearly instantly, which aids learning: www.writelatex.com

So far, LaTeX seems to be different from any markup language I’m familiar with. It’s really hammering home the point that it’s a typesetting language, which means it’s designed with typesetting concepts in mind. All of its tags, or commands as they’re called, are in the context of typesetting conventions, rather than what I’m generally used to.

For instance:

I’ve never seen these text formatting modifiers referred to as “Family,” “Series,” and “Shape.” I’ve thought of them as fonts and formatting. In addition, I wasn’t aware of nuances like slanted vs. italic.

I’m excited to learn more about LaTeX, though I’m not sure I’ll be able to memorize all of these commands and their arguments. I think I’ll need to find a reference document when I actually need to write LaTeX.