Data Science Digest

As mentioned in our prior article on on dashboard development, framing the problem is a critical first step in your process, and one that will ultimately drive utilization and outcomes if done correctly.

It sounds simple, but it's also equally important to outline and follow a project plan. Doing so will allow you to effectively organize your efforts, keep the project going, and ground you to core goals and objectives.

Migrating your information to a cloud-based infrastructure is a critical first step in digital transformation efforts. Doing so will allow you to scale your technology infrastructure to meet growing demands, eliminate the need to maintain hardware, and allow your company to become more agile.

Moving to the cloud streamlines operations and security, giving you added flexibility and more time to focus on the core of what you do best.

There are a several leading players in cloud-based data infrastructure, including Amazon Web Services (AWS). This overview will provide a reference for what AWS products might best support your efforts and take a look at some of the most valuable options for getting started when working with cloud-based data.

My first year as a data scientist, I witnessed myself and others retyping the same lines of code and retracing our work time and time again. Perhaps some of this did not warrant concern.

After all, how long does it take to type the standard imports,

1

import pandas as pd

1

import numpy as np

1

import matplotlib.pyplot as plt

1

%matplotlib inline

and the like?

Yet there were also plenty of real concerns, as my colleagues and I performed many of the same tasks repeatedly, filling null values, standardizing column names, and creating dummy variables. Shouldn’t we be able to standardize these rote processes and not have to recode the entire preprocessing pipeline every time?

Even worse, sometimes after a day’s worth of exploratory analysis, fruitful insights would surface, only to realize that the Jupyter notebook you’d been working on was a jumbled mess, having jumped around in the notebook repeatedly, fixing errors and rerunning cells. How on earth are you supposed to now repeat that process?

It’s also funny to me that despite proclaiming the immense value of object orientated programming, none of my instructors pointed out how to practically implement such a philosophy into a daily workflow.

I hope this article helps you sidestep the pitfalls many of us have fallen into in order to develop a more productive and sensible workflow.