Helping the Blind See

Image Captioning

It's remarkable how much the visually impaired understand about their environment without ever being able to directly see it, but unfortunately not all desired information is transmitted. This project explores designing an ensemble CNN and LSTM Neural Network that captions images for the visually impaired.

Here are some of the things I've learned:

  • Surprisingly, this project was less of a function of how well I could extract features from my images and more a function of the language within the captions. Since my model was guessing the next word in a given sentence, I need to focus on reducing the number of potential outputs.
  • You can't expect Deep Nets to solve all your woes – there is a lot of finess involved with them.
project url

The Challenges of Image Classification

Image Classification

Before convolutional neural networks came around, detecting an object within an image was a manual process of extracting features and piping them into a classifier. This project explores those classic techniques and compares three feature representations of images.

Here are some of the things I've learned:

  • The more complex the process is to generate a feature descriptor does not equate to a highly predictive feature. It's just complex, that's all.
  • Histogram Oriented Gradients are a beautiful feature descriptor that are incredibly versatile in object detection.
project url

Learning About Developer Culture using Hacker News

Topic Modeling / NLP

Hacker News is an extraordinary source of both technology and business related news that pulls developers from many sectors. This project seeks to explore the common topics discussed within developer culture and understand the topics that most excite Hacker News participants.

Here are some of the things I've learned:

  • One of the dominant topics that are discussed at length are programming languages. It was surprising at first to see this compared to other topcis within CompSci, but eventually it dawned on me that programming languages are number one tools in developers day to day work.
project url