Ahan

Linear Regression

2018-10-06T00:00:00+00:00

Linear Regression — Summarised

There are several online resources to teach yourself skills and this is especially true with computer science fields such as machine learning. One such popular resource is the Introduction to Statistical Learning: with Applications in R. It covers many of the modern and widely used statistical learning algorithms.

This ISLR series will cover the key concepts and highlighted summary of each chapter in the book. This should help provide a high-level notion of each algorithm covered and could be useful for anyone that wants to quickly refresh their memory.

Linear regression provides a “very simple approach to supervised learning” and is “useful for predicting a quantitative response”.

“selective focus photography of graph” by M. B. M. on Unsplash

Simple Linear Regression

As the name suggests, linear regression is a straightforward method of predicting the value Y based off of a single predictor *X *and it assumes an approximately linear relationship between the two represented as follows:

Where β₀ and β₁ are knowns as coefficients or parameters. Once the model coefficients have been learned from the training data, they can be used to make predictions.

It is useful to assess the accuracy of the model by minimizing the “error” produced. One way to measure this error is the *residual sum of squares (RSS) *which is defined as follows:

The least squares approach chooses coefficients to minimize the RSS. Using some calculus, one can show that the minimizers are:

Where the overline above *x *and *y *represents the mean of those respective columns.

It’s easy to expand this model to capture multiple predictors and even non-linear predictors such as exponential or polynomial expressions. However, with increasing number of predictors of higher order one needs to be careful about overfitting the data.

Assessing Accuracy

Residual Standard Error: The RSE is an estimate of the standard error deviation of ε (unavoidable error or noise). This can also be explained as the “average amount that the response will deviate from the true regression line”.

Residual Standard Error

Residual sum of squares

R² Statistic: The R² measure provides more of a “proportion” since it always takes a value between 0 and 1 as compared to RSE which is measured in units of Y.

Total sum of squares

Some important questions to consider while developing a linear regression model:

Which of the predictors in X *are useful in predicting *Y?
How well does the model fit the data?
Given a set of predictor values, what response values should we predict?

As always for more detailed concepts and examples please refer to the book. The book also includes lab assignments where you can implement the models you learn in each chapter in R.

If this post receives a positive response I will continue to summarize other chapters in ISLR which include other useful machine learning models like: K-means Clustering, SVMs as well as Decision-Trees.

Internship

2018-08-22T00:00:00+00:00

Introduction

I had the wonderful oppurtunity to work as a Software Engineering Intern for Viasat at Carlsbad in California this summer. This is the first time I’ve interned for a large multi-national company and it’s been a great experience. This blog post will be about my goals and expectations going into the internship and the invaluable experiences and lessons I gained this summer.

Working Culture

I was really suprised to find that the working environment at a big company like Viasat was fairly relaxed and casual. There was definitely an emphasis on getting the assigned work done in the allotted time; however, there was no restriction on how one achieved that. It was also pleasant to discover that the company had a fairly horizontal heriarchy where even the ideas and opinions of interns were well recieved and given importance.

I was assigned a workspace in a fishbowl, a lab with workstations and two glass walls, alongside about a dozen other interns. All of us were working on somewhat similar projects associated with Broadband Communications (BBC) . The supervisor for my project worked nearby and was very available and easy to reach out to through messenger. It was re-assuring to be able to sync up with him and remove blockers with his help at least once everyday.

It was also a very different coping with working a full time job, 40-hours per week, and living on a 8 am - 5 pm schedule. I really enjoyed the fact that after you left work and got home you were finished for the day and you could completely relax. This was verymuch unlike going to university where the lines between studying and relaxing were often blurred. It was easy to see why people found it comfortable to get adjusted to this kind of living.

Technical Learnings

The project I was assigned to involved creating a Zero Touch Provisioning (ZTP) system to automate the deployment and provisioning for the optical fiber network for the newest Viasat satellite Viasat-3 (also known as the Mythical Beast). Most of my work involved networking and software development in python.

One of the most significant technical learnings arose from the orchestration server that I was responsible for developing. The orchestration server was required to execute a long multistep workflow. I was recommended to use a flask server since we would not need to scale by very much and we just needed some simple endpoint functionality.

# Create an endpoint that executes the workflow
@app.route('/endpoint', methods=['GET'])
def func():
  # step 1
  stepOne()

  # step 2
  stepTwo()

  # return from endpoint
  return {'message': 'Success'}

The problem with this design was that each step took a large amount of time to execute. Therefore, the total workflow approximated to about ~ 10 mins. That kind of time to complete a request would be considered bad design and would leave the caller hanging.

To solve this problem I needed to make the actual workflow asynchronous so that the endpoint could return immediately. However, this wasn’t easy because according to the official documentation “Flask is just not designed for large applications or asynchronous servers.”

After doing some research I found a really useful python library called Celery.

# Define a Celery task
@app.task(name="tasks.workflow")
def workflow(x, y):
    z = x + y

    # step 1
    stepOne()

    return z

# Create an endpoint that executes the workflow
@app.route('/endpoint', methods=['GET'])
def func():
  # execute workflow asynchronously
  workflow.apply_async(args=[arg1, arg2])

  # return from endpoint
  return {'message': 'Success'}

As we can see now with the help of Celery our endpoint can finish execution almost immedaitely and our workflow can be executed by another thread in the background. Now you might be wondering how one would obtain the result or status of the workflow that has finished execution.

# Create an endpoint that checks workflow status
@app.route('/status/<taskId>', methods=['GET'])
def func(taskId):
  # Get task object from id
  task = workflow.AsyncResult(taskId)

  # Check status
  if task.state == 'PENDING':
    # Workflow has not yet started
    
  elif task.state != 'FAILURE':
    # Workflow succesfully completed or is in progress
    # Do something with state and result

  else:
    # Something went wrong in the background job
    response = {
        'state': task.state,
        ...
    }

    return jsonify(response)

If you are interested in more details this is the (slightly outdated) blog post that I used as reference while developing the flask-celery orchestrator server for the required workflow: Using Celery With Flask.

Another interesting learning experience was setting up a PostgreSQL database from scratch on a remote CentOS server. This article came in very handy as I configured the server and database priviledges and settings for production: Configure and Install PostgreSQL for CentOS.

Lastly, bash scripting very much handy for automation various parts of the process especially workflow testing. I was able to learn more about and use the very useful terminal tool: tmux.

Soft Skills

It was a wonderful learning experience working with a team of complete strangers. Learning how to communicate most effectively with your team and being a valuable team player was more challenging than I expected. I noted that the team dynamic was certainly variable and was largely dependent on the supervisor that you were assigned to.

I was lucky enough to have a very friendly and personable supervisor. He did a great job introducing the project and he setup “tech-talks” with other members on our team so that we could learn about various technical concepts required for the project.

Another great skill I learned was how to connect with other interns and full time employees at the company. Meeting and conversing with them outside of work was a great way of getting to know people better. It was very useful asking them about their experience at Viasat and about working in the software engineering industry.

Another crucial part was learning how to talk about the project that you’ve worked on. It was pretty incredible being able to pitch our hackathon idea to the CEO of Viasat, Mark Dankburg at the intern hackathon. Similarly, it was nerve-racking but valuable experience demoing and presenting out project to our entire development team. I am sure these skills will prove to be very useful regardless of whether I pursue a more research-oriented or a more implementation-skewed career.

Since I was going to be spending the majority of my summer interning for Viasat I was hoping for a good social atmosphere even outside of work. However, Viasat and my fellow interns blew whatever expectations I had out of the water.

Viasat did an excellent job keeping its interns busy with interesting events almost every weekend. There were many intern events, many organized and paid for by the company and some put together by the social chair among the interns (shoutout to my friends Riya and Pat).

My favorite events were definitely the day at the lagoon when the interns got the opputunity to use several boats, jet-skis and platoons. Not to mention the free Board & Brew food and refreshments that were also provided. I also really enjoyed participating in the Vollyball Tournament where we were assigned to a team of interns, new grads and full-times based off of self-scored skill-level. It gave us a great oopurtunity to network as well as learn/practice volleyball even for people (like me) who had never played before. Perhaps I enjoyed volleyball more than my fellow interns because I had a big advantage because of my abnormally large height.

The other event I really liked was the Viasat intern Hackathon for which they flew in 250+ interns from all over the country to participate. I really enjoy going to hackathons and innovating so, naturally, I could not miss this event. My team “The Slackers” was fortunate enough to win the prize for the category “Capacity to Deliver the Unexpected”. We built the Device Optimization Tool (DOT) that was essentially a more powerful version of Task Manager that tracked your CPU and bandwidth usage and allowed you to set thresholds and notifications for both. We even made an alexa skill as a means to notify you if you were away from your device or were not as tech-savy.

I was also fortunate enough to have some great people working alongside me at the fishbowl. It was suprising how we all got so close over a short-period of time. It was never a dull moment while working in the fishbowl. We would always be playing videos on Netflix on our three screen display in the front of the room and we would also make Costco runs to get snacks for our internal snack table. This ended up being great way to keep our rival fishbowl interns jealous of our setup.

All in all, it was definitely a blast meeting a lot of new people and making some meaningful friendships with many of them. I am certainly going to miss seeing them around. Hopefully I will get a chance to see some of them again.

Conclusion

I think this internship was most benificial to me because it gave me a great insight into what a software engineering job felt like and whether it was something I want to be doing for a living. It also gave me a lot to think about in terms of what kind of work culture I enjoy and see myself perservering in. I have a lot to thank Viasat for in addition to my supervisor and fellow interns.

I was fortunate enough to make it to the Viasat Corporate Blog so hopefully I made my mark at Viasat. I can’t wait till next summer for my next internship!

Example content

2018-03-29T00:00:00+00:00

Howdy! This is an example blog post that shows several types of HTML content supported in this theme.

Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Aenean eu leo quam. Pellentesque ornare sem lacinia quam venenatis vestibulum. Sed posuere consectetur est at lobortis. Cras mattis consectetur purus sit amet fermentum.

Curabitur blandit tempus porttitor. Nullam quis risus eget urna mollis ornare vel eu leo. Nullam id dolor id nibh ultricies vehicula ut id elit.

Etiam porta sem malesuada magna mollis euismod. Cras mattis consectetur purus sit amet fermentum. Aenean lacinia bibendum nulla sed consectetur.

Inline HTML elements

HTML defines a long list of available inline tags, a complete list of which can be found on the Mozilla Developer Network.

To bold text, use <strong>.
To italicize text, use <em>.
Abbreviations, like HTML should use <abbr>, with an optional title attribute for the full phrase.
Citations, like — Mark otto, should use <cite>.
~~Deleted~~ text should use <del> and inserted text should use <ins>.
Superscript ^text uses <sup> and subscript _text uses <sub>.

Most of these elements are styled by browsers with few modifications on our part.

Heading

Vivamus sagittis lacus vel augue rutrum faucibus dolor auctor. Duis mollis, est non commodo luctus, nisi erat porttitor ligula, eget lacinia odio sem nec elit. Morbi leo risus, porta ac consectetur ac, vestibulum at eros.

Code

Cum sociis natoque penatibus et magnis dis code element montes, nascetur ridiculus mus.

// Example can be run directly in your JavaScript console


// Create a function that takes two arguments and returns the sum of those arguments

var adder = new Function("a", "b", "return a + b");

// Call the function

adder(2, 6);
// > 8

Aenean lacinia bibendum nulla sed consectetur. Etiam porta sem malesuada magna mollis euismod. Fusce dapibus, tellus ac cursus commodo, tortor mauris condimentum nibh, ut fermentum massa.

Lists

Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Aenean lacinia bibendum nulla sed consectetur. Etiam porta sem malesuada magna mollis euismod. Fusce dapibus, tellus ac cursus commodo, tortor mauris condimentum nibh, ut fermentum massa justo sit amet risus.

Praesent commodo cursus magna, vel scelerisque nisl consectetur et.
Donec id elit non mi porta gravida at eget metus.
Nulla vitae elit libero, a pharetra augue.

Donec ullamcorper nulla non metus auctor fringilla. Nulla vitae elit libero, a pharetra augue.

Vestibulum id ligula porta felis euismod semper.
Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.
Maecenas sed diam eget risus varius blandit sit amet non magna.

Cras mattis consectetur purus sit amet fermentum. Sed posuere consectetur est at lobortis.

HyperText Markup Language (HTML): The language used to describe and define the content of a Web page
Cascading Style Sheets (CSS): Used to describe the appearance of Web content
JavaScript (JS): The programming language used to build advanced Web sites and applications

Integer posuere erat a ante venenatis dapibus posuere velit aliquet. Morbi leo risus, porta ac consectetur ac, vestibulum at eros. Nullam quis risus eget urna mollis ornare vel eu leo.

Images

Quisque consequat sapien eget quam rhoncus, sit amet laoreet diam tempus. Aliquam aliquam metus erat, a pulvinar turpis suscipit at.

Tables

Aenean lacinia bibendum nulla sed consectetur. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Name	Upvotes	Downvotes
Totals	21	23
Alice	10	11
Bob	4	3
Charlie	7	9

Nullam id dolor id nibh ultricies vehicula ut id elit. Sed posuere consectetur est at lobortis. Nullam quis risus eget urna mollis ornare vel eu leo.

Want to see something else added? Open an issue.

Ahan Mukhopadhyay

2018-03-22T00:00:00+00:00

Welcome to my blog-centric personal website! This first post is a little about me. To check out some of my other posts click on the blog link on the sidebar!

I am a second year undergraduate at UC San Diego pursuing a Bachelor of Science in Computer Science and a minor in Economics. I am a tutor for the CSE Department and a sustainability intern at the Office of Sustainability.

Experience

Viasat
Software Enginering Intern
Carlsbad, CA // Summer 2018

Designed a ZTP system for automating the deployment of next-gen network
Implemented an asynchronous Orchestrator for long workflow tasks
Created a VPN concentrator that uses the daemon charon to detect ipsec tunnels
Used kubernetes to deploy multiple docker-image pods to the cloud
Won intern hackathon category Delivering The Unexpected
Tech Used: Python, Flask, Celery, Docker, Kubernetes, PostgresSQL

CSE Department
Tutor
La Jolla, CA // Sept 2017 - Mar 2018

Graded programming assignments, quizzes and exams
Hosted lab hours to resolve student questions and bugs
Led discussion sections and made useful slides
Proctored quizzes and exams in class

Office of Sustainability
Green Events Intern
La Jolla, CA // Jan - Mar 2018

#MyLastTrash is a University of California initiative that connects students to the 2020 zero waste goal and provides support for projects that approach this target.
Ahan is responsible for developing the Green Events Program that endorses greener campus-wide student organization and staff hosted events.
The program seeks to establish a comprehensive guideline for planning Green Events to achieve a win-win situation by reducing event costs and waste generation.
He is also working on a Green Event Certification that will certify organizations fully capable of autonomously hosting Green Events.
Ahan hopes that the Green Events Program will pave the way to zero waste events and bring the UC San Diego campus much closer to it’s zero waste goal.

Swish
Software Enginering Intern
Remote // Jul - Sept 2017

Worked on all aspects of backend web development
Improved concurrency in the room8 (iOS app) server for 10k active users
Created REST APIs to expand server support to mobile apps
Created Gatorade Loyalty CMS server and contributed to Gatorade.com
Tech Used: NodeJs, ReactJs, MeteorJS, KeystoneJS, MongoDB

Yobi Technologies
Software Development Intern
Gurgaon, India // Apr - Jun 2017

Developed an extreme weather alert tool for use by the Government.
Product has been developed further to scale with combined populations 20 mil.
Created a web app that collected data from 100+ IoT weather stations.
Implemented data visualization models that helped identify at-risk regions.
Made a MVP of a low-cost CCTV that could count traffic, piloted by police
Tech Used: Python, Flask, MongoDB, OpenCV, Plotly, Raspi

Projects

ChessBox – Physical chessboard AI
PiB - Project in a Box // Jan - Mar 2018

AI obtained a 1400-1500 chess.com competitive rating
Employs Alpha-Beta Pruning and Iterative deeping for search.
Percieved instantaneous depth-7 search computation.
Avoids the horizon effect using a Quiscience search.
Interacts with physical board using an interface.
Chessboard uses IR sensors to detect piece movement.
Tech Used: C/C++, AI, AlphaBeta Pruning

ClickBlock – Chrome extension that counters clickbaits
Social Hack Winner HackUCI 2017

Automatically checks every link on your FB feed for clickbait.
Changes title a more appropriate one for the article's content.
Provides a 10-line summary on hover and a probability of correctness.
Classifier trained on 5000 articles with 89% tested accuracy.
Published on Chrome Web Store.
Tech Used: Python, scikit-learn, beautifulsoup, Flask

Trivents – Events Management Project
DocuSign Prize Winner SD Hacks 2016

A platform that allows event posting to all social media pages.
Markets events to people in the neighborhood.
Each event has a QR code which can be used to signup for the event
Messenger Bot uses the QR code to register you for the event.
QR ticket scanning helps prevent longs queues.
Tech Used: NodeJS, MeteorJS, Messenger Bot, FB Graph API, GoogleMaps API

Elephant Project – Saving Elephants Using Ultrasound
Research Project 2015

Targets elephants that die from crossing railroads in India.
Made an infrasounds generator to alert Elphants from a distance.
Program adjusts for Doppler Effect with input velocity
Tested its use and wrote a comprehensive research paper.
Presented prototype at multiple conferences.
Tech Used: Java, LateX

Teaching Assistant Positions

CSE 11 - Object Oriented Programming
CSE 30 - Computer Organization and Systems Programming

Relevant Coursework (at UC San Diego)

CSE 105 - Theory of Computability
CSE 101 - Design and Analysis of Algorithms
CSE 100 - Advanced Data Structures
CSE 95 - Tutor Apprenticeship
CSE 30 - Computer Organization and Systems Programming
CSE 21 - Mathematics for Algorithms and Systems
CSE 20 - Discrete Mathematics
CSE 15L - Software Tools & Techniques
CSE 12 - Data Strcutures & OOP Design
CSE 11 - Introduction to Object Oriented Programming