EVALUATION OF TOPIC MODELING: TOPIC COHERENCE In this article, we will go through the evaluation of Topic Modelling by introducing the concept of Topic coherence, as topic models give no guaranty on the interpretability of their output. Topic modeling provides us with methods to organize, understand and summarize large collections of textual information. There are many techniques that are

used to

PROCESSING HUGE DATASET WITH PYTHON Processing Huge Dataset with Python. This tutorial introduces the processing of a huge dataset in python. It allows you to work with a big quantity of data with your own laptop. With this method, you could use the aggregation functions on a dataset that you cannot import in a DataFrame. In our example, the machine has 32 cores with 17GB of Ram. INTERPRETATION OF THE AUC MACHINE LEARNING IN EXCEL WITH PYTHON Machine learning is an important topic in lots of industries right now. It’s a fast moving field with lots of active research and receives huge amounts of media attention. This post isn’t intended to be an introduction to machine learning, or a comprehensive overview of the state of the art. Instead it PRINCIPAL COMPONENT ANALYSIS (PCA) WITH PYTHON Principal Component Analysis(PCA) is an unsupervised statistical technique used to examine the interrelation among a set of variables in order to identify the underlying structure of those variables. In simple words, suppose you have 30 features column in a data frame so it will help to reduce the number of features making a new feature WHAT’S THE INTUITION BEHIND CONTINUOUS NAIVE BAYESSEE MORE ON

DATASCIENCEPLUS.COM

GRADIENT BOOSTING IN R Gradient boosting generates learners using the same general boosting learning process. Gradient boosting identifies hard examples by calculating large residuals-\( (y_{actual}-y_{pred} ) \) computed in the previous iterations.Now for the training examples which had large residual values for \(F_{i-1}(X) \) model,those examples will be the training examples for the next \(F_i(X)\) Model.It MULTICOLLINEARITY IN R Multicollinearity in R. One of the assumptions of Classical Linear Regression Model is that there is no exact collinearity between the explanatory variables. If the explanatory variables are perfectly correlated, you will face with these problems: However, the TIME SERIES ANALYSIS USING ARIMA MODEL IN R HOW TO EXPORT REGRESSION RESULTS FROM R TO MS WORD In this post, I will present a simple way how to export your regression results (or output) from R into Microsoft Word. Previously, I have written a tutorial how to create Table 1 with study characteristics and to export into Microsoft Word. These posts are especially useful for researchers who prepare their manuscript for publication in peer-reviewed journals. EVALUATION OF TOPIC MODELING: TOPIC COHERENCE In this article, we will go through the evaluation of Topic Modelling by introducing the concept of Topic coherence, as topic models give no guaranty on the interpretability of their output. Topic modeling provides us with methods to organize, understand and summarize large collections of textual information. There are many techniques that are

used to

PROCESSING HUGE DATASET WITH PYTHON Processing Huge Dataset with Python. This tutorial introduces the processing of a huge dataset in python. It allows you to work with a big quantity of data with your own laptop. With this method, you could use the aggregation functions on a dataset that you cannot import in a DataFrame. In our example, the machine has 32 cores with 17GB of Ram. INTERPRETATION OF THE AUC MACHINE LEARNING IN EXCEL WITH PYTHON Machine learning is an important topic in lots of industries right now. It’s a fast moving field with lots of active research and receives huge amounts of media attention. This post isn’t intended to be an introduction to machine learning, or a comprehensive overview of the state of the art. Instead it PRINCIPAL COMPONENT ANALYSIS (PCA) WITH PYTHON Principal Component Analysis(PCA) is an unsupervised statistical technique used to examine the interrelation among a set of variables in order to identify the underlying structure of those variables. In simple words, suppose you have 30 features column in a data frame so it will help to reduce the number of features making a new feature WHAT’S THE INTUITION BEHIND CONTINUOUS NAIVE BAYESSEE MORE ON

DATASCIENCEPLUS.COM

GRADIENT BOOSTING IN R Gradient boosting generates learners using the same general boosting learning process. Gradient boosting identifies hard examples by calculating large residuals-\( (y_{actual}-y_{pred} ) \) computed in the previous iterations.Now for the training examples which had large residual values for \(F_{i-1}(X) \) model,those examples will be the training examples for the next \(F_i(X)\) Model.It MULTICOLLINEARITY IN R Multicollinearity in R. One of the assumptions of Classical Linear Regression Model is that there is no exact collinearity between the explanatory variables. If the explanatory variables are perfectly correlated, you will face with these problems: However, the TIME SERIES ANALYSIS USING ARIMA MODEL IN R HOW TO EXPORT REGRESSION RESULTS FROM R TO MS WORD In this post, I will present a simple way how to export your regression results (or output) from R into Microsoft Word. Previously, I have written a tutorial how to create Table 1 with study characteristics and to export into Microsoft Word. These posts are especially useful for researchers who prepare their manuscript for publication in peer-reviewed journals. PRINCIPAL COMPONENT ANALYSIS (PCA) WITH PYTHON Principal Component Analysis(PCA) is an unsupervised statistical technique used to examine the interrelation among a set of variables in order to identify the underlying structure of those variables. In simple words, suppose you have 30 features column in a data frame so it will help to reduce the number of features making a new feature USING CACHE TO AVOID RE-PROCESSING, IMPROVE UX, AND If you’ve never heard of cache (/kaSH/) before, Google it and you’ll quickly find that it is “a collection of items of the same type stored in a hidden or inaccessible place”. Basically, you have “something” stored “somewhere” so you can fetch it “sometime” later. If INTERPRETATION OF THE AUC Interpretation of the AUC. The AUC* or concordance statistic c is the most commonly used measure for diagnostic accuracy of quantitative tests. It is a discrimination measure which tells us how well we can classify patients in two groups: those with and those without the outcome of interest. Since the measure is based on ranks, it is not TIME SERIES ANALYSIS USING ARIMA MODEL IN R Time series data are data points collected over a period of time as a sequence of time gap. Time series data analysis means analyzing the available data to find out the pattern or trend in the data to predict some future values which will, in turn, help more effective and optimize business decisions. HOW TO BUILD A SIMPLE FLOWCHART WITH R: DIAGRAMMER PACKAGE After some search, I found that there are a few packages in R which allow making exemplary flowcharts. The one which I found easy to use was DiagrammeR. The advantage of this packages is that generate diagrams using code within R Markdown syntax. Load the library. library (DiagrammeR) Copy. Let start with a simple example and go

through the code.

UNDERSTANDING THE COVARIANCE MATRIX With the covariance we can calculate entries of the covariance matrix, which is a square matrix given by Ci, j = σ(xi, xj) where C ∈ Rd × d and d describes the dimension or number of random variables of the data (e.g. the number of features like height, width, weight, ). Also the covariance matrix is symmetric since σ(xi, xj) = σ(xj, xi). LINEAR REGRESSION WITH HEALTHCARE DATA FOR BEGINNERS IN R The model. I will use the function lm () to create a linear regression model. In the first model I will not adjust for confunders, insted, I will do a univariate model. fit1 % spread (Year, BMI) wide ## ID Name 2012 2013 2014 ## 1 1 Dora 24 34 19 ## 2 2 John 33 27 25 ## 3 3 Rob 33 27 34 Copy. Below, I will convert dataset from wide to long with gather function, in which I include variables I like to put in one column as Year, and besides that the column with GRADIENT BOOSTING IN R Gradient boosting generates learners using the same general boosting learning process. Gradient boosting identifies hard examples by calculating large residuals-\( (y_{actual}-y_{pred} ) \) computed in the previous iterations.Now for the training examples which had large residual values for \(F_{i-1}(X) \) model,those examples will be the training examples for the next \(F_i(X)\) Model.It INTERPRETATION OF THE AUC EVALUATION OF TOPIC MODELING: TOPIC COHERENCE In this article, we will go through the evaluation of Topic Modelling by introducing the concept of Topic coherence, as topic models give no guaranty on the interpretability of their output. Topic modeling provides us with methods to organize, understand and summarize large collections of textual information. There are many techniques that are

used to

MACHINE LEARNING IN EXCEL WITH PYTHON Machine learning is an important topic in lots of industries right now. It’s a fast moving field with lots of active research and receives huge amounts of media attention. This post isn’t intended to be an introduction to machine learning, or a comprehensive overview of the state of the art. Instead it HOW TO GET STOCK PRICES AND PLOT THEM? Stocks is an xts object with an index column for date reference and 3 columns for adjusted stock prices. Now is required to do two additional transformations to stocks before plotting. 1) Assigning names to the columns given that at this point ggplot2 will read them as 1, 2, and 3. So first I assign the bank name, then 2) define the

index

USING MONGODB WITH R PERFORMING SQL SELECTS ON R DATA FRAMES Performing SQL selects on R data frames. For anyone who has SQL background and who wants to learn R, I guess the sqldf package is very useful because it enables us to use SQL commands in R. One who has basic SQL skills can manipulate data frames in R using their SQL skills. You can read more about sqldf package from cran. PRINCIPAL COMPONENT ANALYSIS (PCA) WITH PYTHON Principal Component Analysis(PCA) is an unsupervised statistical technique used to examine the interrelation among a set of variables in order to identify the underlying structure of those variables. In simple words, suppose you have 30 features column in a data frame so it will help to reduce the number of features making a new feature EXTRACTING TABLES FROM PDFS IN R USING THE TABULIZER Recently I wanted to extract a table from a pdf file so that I could work with the table in R. Specifically, I wanted to get data on layoffs in California from the California Employment Development Department.The EDD publishes a list of all of the layoffs in the state that fall under the WARN act here.Unfortunately, the tables are available only in pdf format. CONVERTING DATA FROM LONG TO WIDE SIMPLIFIED: TIDYVERSE Transform dataset from long to wide. wide = longdata1 %>% spread (Year, BMI) wide ## ID Name 2012 2013 2014 ## 1 1 Dora 24 34 19 ## 2 2 John 33 27 25 ## 3 3 Rob 33 27 34 Copy. Below, I will convert dataset from wide to long with gather function, in which I include variables I like to put in one column as Year, and besides that the column with USING CACHE TO AVOID RE-PROCESSING, IMPROVE UX, AND If you’ve never heard of cache (/kaSH/) before, Google it and you’ll quickly find that it is “a collection of items of the same type stored in a hidden or inaccessible place”. Basically, you have “something” stored “somewhere” so you can fetch it “sometime” later. If INTERPRETATION OF THE AUC Interpretation of the AUC. The AUC* or concordance statistic c is the most commonly used measure for diagnostic accuracy of quantitative tests. It is a discrimination measure which tells us how well we can classify patients in two groups: those with and those without the outcome of interest. Since the measure is based on ranks, it is not PRINCIPAL COMPONENT ANALYSIS (PCA) WITH PYTHON Principal Component Analysis(PCA) is an unsupervised statistical technique used to examine the interrelation among a set of variables in order to identify the underlying structure of those variables. In simple words, suppose you have 30 features column in a data frame so it will help to reduce the number of features making a new feature CACHE | DATASCIENCE+ An online community for showcasing R & Python tutorials. Category

cache. 1 articles

MACHINE LEARNING FOR DIABETES WITH PYTHON The Data. The diabetes data set was originated from UCI Machine Learning Repository and can be downloaded from here. The diabetes data set consists of 768 data points, with 9 features each: “Outcome” is the feature we are going to predict, 0 means No diabetes, 1 means diabetes. Of these 768 data points, 500 are labeled as 0 and 268 as 1: UNDERSTANDING THE COVARIANCE MATRIX With the covariance we can calculate entries of the covariance matrix, which is a square matrix given by Ci, j = σ(xi, xj) where C ∈ Rd × d and d describes the dimension or number of random variables of the data (e.g. the number of features like height, width, weight, ). Also the covariance matrix is symmetric since σ(xi, xj) = σ(xj, xi). HOW TO GET STOCK PRICES AND PLOT THEM? Stocks is an xts object with an index column for date reference and 3 columns for adjusted stock prices. Now is required to do two additional transformations to stocks before plotting. 1) Assigning names to the columns given that at this point ggplot2 will read them as 1, 2, and 3. So first I assign the bank name, then 2) define the

index

HOW TO COMPARE DISTRIBUTION BY USING DENSITY PLOTS IN R The function we use for making the density plot is sm.density.compare () from sm package. To install and load the package use the code below: In this example, I am using iris data set and comparing the distribution of the length of sepal for different species. After you load the dataset run the code below to build the density plot. HOW TO DEAL WITH MISSING VALUES IN R It might happen that your dataset is not complete, and when information is not available we call it missing values. In R the missing values are coded by the symbol NA. To identify missings in your dataset the function is is.na(). First lets create a small dataset: Name % spread (Year, BMI) wide ## ID Name 2012 2013 2014 ## 1 1 Dora 24 34 19 ## 2 2 John 33 27 25 ## 3 3 Rob 33 27 34 Copy. Below, I will convert dataset from wide to long with gather function, in which I include variables I like to put in one column as Year, and besides that the column with INTERPRETATION OF THE AUC Interpretation of the AUC. The AUC* or concordance statistic c is the most commonly used measure for diagnostic accuracy of quantitative tests. It is a discrimination measure which tells us how well we can classify patients in two groups: those with and those without the outcome of interest. Since the measure is based on ranks, it is not USING CACHE TO AVOID RE-PROCESSING, IMPROVE UX, AND If you’ve never heard of cache (/kaSH/) before, Google it and you’ll quickly find that it is “a collection of items of the same type stored in a hidden or inaccessible place”. Basically, you have “something” stored “somewhere” so you can fetch it “sometime” later. If PRINCIPAL COMPONENT ANALYSIS (PCA) WITH PYTHON Principal Component Analysis(PCA) is an unsupervised statistical technique used to examine the interrelation among a set of variables in order to identify the underlying structure of those variables. In simple words, suppose you have 30 features column in a data frame so it will help to reduce the number of features making a new feature CACHE | DATASCIENCE+ An online community for showcasing R & Python tutorials. Category

cache. 1 articles

MACHINE LEARNING FOR DIABETES WITH PYTHON The Data. The diabetes data set was originated from UCI Machine Learning Repository and can be downloaded from here. The diabetes data set consists of 768 data points, with 9 features each: “Outcome” is the feature we are going to predict, 0 means No diabetes, 1 means diabetes. Of these 768 data points, 500 are labeled as 0 and 268 as 1: UNDERSTANDING THE COVARIANCE MATRIX With the covariance we can calculate entries of the covariance matrix, which is a square matrix given by Ci, j = σ(xi, xj) where C ∈ Rd × d and d describes the dimension or number of random variables of the data (e.g. the number of features like height, width, weight, ). Also the covariance matrix is symmetric since σ(xi, xj) = σ(xj, xi). HOW TO GET STOCK PRICES AND PLOT THEM? Stocks is an xts object with an index column for date reference and 3 columns for adjusted stock prices. Now is required to do two additional transformations to stocks before plotting. 1) Assigning names to the columns given that at this point ggplot2 will read them as 1, 2, and 3. So first I assign the bank name, then 2) define the

index

HOW TO COMPARE DISTRIBUTION BY USING DENSITY PLOTS IN R The function we use for making the density plot is sm.density.compare () from sm package. To install and load the package use the code below: In this example, I am using iris data set and comparing the distribution of the length of sepal for different species. After you load the dataset run the code below to build the density plot. HOW TO CREATE, RENAME, RECODE AND MERGE VARIABLES IN R How to Create, Rename, Recode and Merge Variables in R. To create a new variable or to transform an old variable into a new one, usually, is a simple task in R. The common function to use is newvariable % spread (Year, BMI) wide ## ID Name 2012 2013 2014 ## 1 1 Dora 24 34 19 ## 2 2 John 33 27 25 ## 3 3 Rob 33 27 34 Copy. Below, I will convert dataset from wide to long with gather function, in which I include variables I like to put in one column as Year, and besides that the

column with

INTERPRETATION OF THE AUC Interpretation of the AUC. The AUC* or concordance statistic c is the most commonly used measure for diagnostic accuracy of quantitative tests. It is a discrimination measure which tells us how well we can classify patients in two groups: those with and those without the outcome of interest. Since the measure is based on ranks, it is not USING CACHE TO AVOID RE-PROCESSING, IMPROVE UX, AND If you’ve never heard of cache (/kaSH/) before, Google it and you’ll quickly find that it is “a collection of items of the same type stored in a hidden or inaccessible place”. Basically, you have “something” stored “somewhere” so you can fetch it “sometime” later. If PRINCIPAL COMPONENT ANALYSIS (PCA) WITH PYTHON Principal Component Analysis(PCA) is an unsupervised statistical technique used to examine the interrelation among a set of variables in order to identify the underlying structure of those variables. In simple words, suppose you have 30 features column in a data frame so it will help to reduce the number of features making a new feature CACHE | DATASCIENCE+ An online community for showcasing R & Python tutorials. Category

cache. 1 articles

MACHINE LEARNING FOR DIABETES WITH PYTHON The Data. The diabetes data set was originated from UCI Machine Learning Repository and can be downloaded from here. The diabetes data set consists of 768 data points, with 9 features each: “Outcome” is the feature we are going to predict, 0 means No diabetes, 1 means diabetes. Of these 768 data points, 500 are labeled as 0 and 268 as 1: UNDERSTANDING THE COVARIANCE MATRIX With the covariance we can calculate entries of the covariance matrix, which is a square matrix given by Ci, j = σ(xi, xj) where C ∈ Rd × d and d describes the dimension or number of random variables of the data (e.g. the number of features like height, width, weight, ). Also the covariance matrix is symmetric since σ(xi, xj) = σ(xj, xi). HOW TO GET STOCK PRICES AND PLOT THEM? Stocks is an xts object with an index column for date reference and 3 columns for adjusted stock prices. Now is required to do two additional transformations to stocks before plotting. 1) Assigning names to the columns given that at this point ggplot2 will read them as 1, 2, and 3. So first I assign the bank name, then 2) define the

index

HOW TO COMPARE DISTRIBUTION BY USING DENSITY PLOTS IN R The function we use for making the density plot is sm.density.compare () from sm package. To install and load the package use the code below: In this example, I am using iris data set and comparing the distribution of the length of sepal for different species. After you load the dataset run the code below to build the density plot. HOW TO CREATE, RENAME, RECODE AND MERGE VARIABLES IN R How to Create, Rename, Recode and Merge Variables in R. To create a new variable or to transform an old variable into a new one, usually, is a simple task in R. The common function to use is newvariable