Gopal Krishna Ranjan

Gopal is a passionate Data Engineer and Data Analyst. He has implemented many end to end solutions using Big Data, Machine Learning, OLAP, OLTP, and cloud technologies. He loves to share his experience at https://sqlrelease.com//. Connect with Gopal on LinkedIn at https://www.linkedin.com/in/ergkranjan/.

RDD, DataFrame, and DataSet – Introduction to Spark Data Abstraction

Leave a Comment / Spark / / /

Apache Spark is a general purpose distributed computing engine used for Big Data processing – Batch and stream processing. It provides high level APIs like Spark SQL, Spark Streaming, MLib, and GraphX to allow interaction with core functionalities of Apache Spark. Spark also facilitates several core data abstractions on top of the distributed collection of […]

RDD, DataFrame, and DataSet – Introduction to Spark Data Abstraction Read More »

Big Data processing using Apache Spark – Introduction

Leave a Comment / Spark / / / ,

What is Spark Apache spark is an open source general purpose distributed cluster computing framework. It is an unified computing engine for big data processing. Spark is designed for lightning fast cluster computing especially for fast computation. An application can run up to 100 times faster than Hadoop MapReduce using Spark in-memory cluster computing. Also,

Big Data processing using Apache Spark – Introduction Read More »

Understanding Map join in Hive

Leave a Comment / Hadoop, Hive / / / , , , ,

Apache Hive is a big data query language which is used to read, transform and write large datasets in a distributed environment. It has a SQL like syntax which gets translated into a MapReduce job in order to execute on Hadoop clusters. In Hadoop ecosystem, we use Hive for batch processing to extract, transform and

Understanding Map join in Hive Read More »

Python use case – Save each worksheet as a separate excel workbook

1 Comment / Python / / / , ,

In this post “Python use case – Save each worksheet as a separate excel workbook“, we are going to learn that how we can create a separate workbook for each worksheet of a given excel file. We will be copying data, values, formatting and all other settings of the sheet in the newly created workbook.

Python use case – Save each worksheet as a separate excel workbook Read More »

Building Decision Tree model in python from scratch – Step by step

1 Comment / Data Science, Machine Learning, Python / / / , , ,

In previous post, we created our first Machine Learning model using Logistic Regression to solve a classification problem. We used “Wisconsin Breast Cancer dataset” for demonstration purpose. Now, in this post “Building Decision Tree model in python from scratch – Step by step”, we will be using IRIS dataset which is a standard dataset that

Building Decision Tree model in python from scratch – Step by step Read More »

Building first Machine Learning model using Logistic Regression in Python – Step by Step

Leave a Comment / Data Science, Machine Learning, Python / / / , ,

This post briefs how to create our first machine learning predictive model using Logistic regression in Python. When we start working on a Machine Learning project, first, we perform some data wrangling and transformation to get the tidy dataset. Then, we perform some EDA to find trends, patterns, and outliers in the given dataset. Once, we have machine-interpretable data

Building first Machine Learning model using Logistic Regression in Python – Step by Step Read More »

Exploratory Data Analysis (EDA) using Python – Second step in Data Science and Machine Learning

2 Comments / Analytics/ML, Data Analysis, Data Science, Machine Learning, Python / / / , , , , ,

In the previous post, “Tidy Data in Python – First Step in Data Science and Machine Learning”, we discussed the importance of the tidy data and its principles. In a Machine Learning project, once we have a tidy dataset in place, it is always recommended to perform EDA (Exploratory Data Analysis) on the underlying data

Exploratory Data Analysis (EDA) using Python – Second step in Data Science and Machine Learning Read More »

Quick guide to Bash commands for Big Data Analysis

1 Comment / Big Data/Cloud / / / ,

In this post “Quick guide to Bash commands for Big Data Analysis”, we are going to explore some basic Bash/Linux commands which are very useful in data analysis. Bash is a command line interpreter for the GNU OS(a UNIX like free OS) which typically runs in a command line window. It accepts the command submitted

Quick guide to Bash commands for Big Data Analysis Read More »

Python use case – Resampling time series data (Upsampling and downsampling) – SQL Server 2017

Leave a Comment / Data Analysis, Machine Learning, Python, SQL Server / / / , , , , ,

Resampling time series data in SQL Server using Python’s pandas library In this post, we are going to learn how we can use the power of Python in SQL Server 2017 to resample time series data using Python’s pandas library. Sometimes, we get the sample data (observations) at a different frequency (higher or lower) than

Python use case – Resampling time series data (Upsampling and downsampling) – SQL Server 2017 Read More »

Tidy Data in Python – First Step in Data Science and Machine Learning

1 Comment / Analytics/ML, Data Analysis, Data Science, Machine Learning, Python / / / , , , ,

Most of the Data Science / Machine Learning projects follow the Pareto principle where we spend almost 80% of the time in data preparation and remaining 20% in choosing and training the appropriate ML model. Mostly, the datasets we get to create Machine Learning models are messy datasets and cannot be fitted into the model

Tidy Data in Python – First Step in Data Science and Machine Learning Read More »