AWS launches SageMaker Studio, a web-based IDE for Machine Learning & Data Science

Navin Bondade Artificial Intelligence, Data Science, Deep Learning, Machine Learning

At the event of re:Invent conference, the Amazon Web Service CEO Andy Jassy announces the release of SageMaker Studio, an integrated development environment for Machine Learning.

The AWS’s SageMaker Studio is a web-based IDE for building and training machine learning workflows. It helps to brings code editing, training, job tracking, tuning, and debugging all into a single web-based interface.

The Amazon Web Service includes everything a data scientist would need to get started in SageMaker Studio, including ways to organize and manage notebooks, data sets, code, and model development and training.

The SageMaker Studio attempts to solve important pain points for Data Scientists and Machine Learning Developers and Engineers by streamlining model training and maintenance workloads.

The SageMaker Studio offers a number of features such as the ability to share projects and folders with others who are working on the same project, including the ability to discuss notebooks and results.

The SageMaker Studio has already integrated most of its features with AWS’s SageMaker Machine Learning service so you can directly train your models and can also automatically scale based on your needs.

In addition to Studio, AWS also today announced a number of other updates to SageMaker that are integrated into Studio. Including Amazon SageMaker Notebooks, Amazon SageMaker Experiments, Amazon SageMaker Autopilot, Amazon SageMaker Debugger, and Amazon SageMaker Model Monitor.

One of my personal favorite features includes the SageMaker Notebooks that lets you quickly spin up a Jupyter notebook for ML projects. The CPU usage with SageMaker Notebooks also managed by AWS.

At its core, the SageMaker Studio is based on JupyterLab, the next-generation interface from Project Jupyter which is the most common environment used by data scientists for exploring data and ML algorithms.

Recommended: No Degree In AI Or Data Science Needed To work In Tesla, Elon Musk Reveals

With the new upgrade, the Amazon Web Service said that the SageMaker now offers long supported notebook instances, which require a user to log on to AWS & provision a virtual machine.

The new offering promises to launch notebooks “in seconds” and supports sharing with multiple users by integrating with AWS’s single-sign-on (SSO) services, allowing users to access notebooks hosted in AWS without requiring AWS-specific credentials.

It provides Jupyter NoteBooks running R/Python kernels with a compute instance that we can choose as per our data engineering requirements on demand.

With that the SageMaker Notebooks attempt to solve the biggest barrier for people learning data science: getting a Python or R environment working and figuring out how to use a notebook.

The studio delivers single-click Notebooks for the SageMaker environment, competing directly against Google Colab or Microsoft Azure Notebooks in the Notebook-as-a-Service category.

The SageMaker Studio includes an integration with the new SageMaker Experiments service.

It is designed to help ML practitioners manage large numbers of related training jobs, this is a problem that arises when searching for hyperparameters that lead to the best-performing model.

The AWS has also introduced hyperparameter-tuning jobs in 2018, SageMaker Experiments provides an abstraction-layer by introducing two core concepts: a trial, which is a training job with a certain configuration and set of hyperparameters, and an experiment, which is a group of related trials.

Another feature that grabs my attention was the SageMaker Autopilot, which follows the same old rules, it automates the creation of machine learning models and automatically chooses algorithms and tunes models.

SageMaker Autopilot can automatically generate & run experiments given only a file containing a dataset.

Autopilot runs data pre-processing and feature-engineering jobs to infer the best model architecture before running hyperparameter-tuning jobs to find the best fit of that model.

“With AutoML, here’s what happens: You send us your CSV file with the data that you want a model for where you can just point to the S3 location and Autopilot does all the transformation of the model to put in a format so we can do machine learning; it selects the right algorithm.”

“Then it trains 50 unique models with a little bit different configurations of the various variables because you don’t know which ones are going to lead to the highest accuracy,” CEO Andy Jassy said onstage at re:Invent.

“Then what we do is we give you in SageMaker Studio a model leaderboard where you can see all 50 models ranked in order of accuracy. And we give you a notebook underneath every single one of these models so that when you open the notebook, it has all the recipe of that particular model.”

If we talk about the SageMaker Experiments, the SageMaker Experiments are for training and tuning models automatically and capture parameters when testing models. Older experiments can be searched for by name, data set to use, or parameters to make it easier to share and search models.

And the SageMaker Debugger is made to improve the accuracy of machine learning models, while SageMaker Model Monitor is a way to detect concept drift.

“With concept drift, what we do is we create a set of baseline statistics on the data in which you train the model and then we actually analyze all the predictions, compare it to the data used to create the model, and then we give you a way to visualize where there appears to be concept drift, which you can see in SageMaker Studio,” Jassy said.

The AWS Sagemaker is a great tool or most data scientists who would want to accomplish a truly end-to-end ML solution. It’s a one-stop-shop for all the machine learning tools and results you need to get started.

It takes care of abstracting a ton of software development skills necessary to accomplish the task while still being highly effective and flexible and cost-effective.

Most importantly, it helps you focus on the core ML experiments and supplements the remainder necessary skills with easy abstracted tools similar to our existing workflow.