There are commercial and open-source options available for solvers also. For a class of optimization problems referred to as Mixed Integer Linear Programs (MILP), the commercial solvers such as CPLEX, and GUROBI perform significantly better than open source solvers such as glpk, and CBC. A new open-source solver HiGHS has been developed recently that has generated quite a bit of buzz and by different accounts looks like a promising option. There is now a highs package in R that can call the HiGHS solver.

In this blog, I wanted to explore using OMPR modeling system with HiGHS solver by using it to solve a few examples of LP/MILP problems.

mdl = MIPModel() %>%
      add_variable(x0, lb = 0, ub = 4, type = "continuous") %>%
      add_variable(x1, lb = 1, type = "continuous") %>%
      set_objective(x0+x1+3, sense = "min") %>%
      add_constraint(x1 <= 7) %>%
      add_constraint(x0 + 2*x1 <= 15) %>%
      add_constraint(x0 + 2*x1 >= 5) %>%
      add_constraint(3*x0 + 2*x1 >= 6)

# solve model
s = mdl %>% solve_model(highs_optimizer())

# get solution
s$status
s$objective_value
s$solution

np = length(plants)
nm = length(mkts)
# create ompr model
mdl = MIPModel() %>%
  add_variable(x[i, j], i=1:np, j=1:nm, type = "continuous",lb = 0) %>%
  # objective: min cost
  set_objective(sum_over(cost(i, j) * x[i, j], i = 1:np, j = 1:nm), sense = "min") %>% 
  # supply from each plant is below capacity
  add_constraint(sum_over(x[i, j], j = 1:nm) <= cap[i], i = 1:np) %>%  
  # supply to each market meets demand
  add_constraint(sum_over(x[i, j], i = 1:np) >= dem[j], j = 1:nm)

# OMPR model
ns = nrow(nodes_df)
nc = 4
edge_str = edge_df %>% mutate(edge_str = glue("{fromid}_{toid}")) %>% pull(edge_str)
mdl = MIPModel()
mdl = mdl %>% add_variable(x[i, c], i = 1:ns, c = 1:nc, type = "integer", lb = 0, ub = 1)
mdl = mdl %>% add_variable(y[c], c = 1:nc, type = "integer", lb = 0, ub = 1)
mdl = mdl %>% set_objective(sum_over(y[c], c=1:nc), sense = "min")
mdl = mdl %>% add_constraint(sum_over(x[i, c], c = 1:nc) == 1, i = 1:ns)
mdl = mdl %>% add_constraint(x[i, c] + x[j, c] <= y[c], i = 1:ns, j = 1:ns, c = 1:nc, glue("{i}_{j}") %in% edge_str)

My goal in this blog is to see how far I can get in terms of using Pyomo from R using the reticulate package. The simplest option would be to develop the model in pyomo and call it from R using reticulate. However, it still requires writing the pyomo model in python. I want to use reticulate to write the pyomo model using R. The details of the blog post (along with code) are in this location.

There are several resources on the web that show animations/illustrations of proofs of mathematical identities and theorems without words (or close to it). I wanted to take a few of those examples and use gganimate to recreate the illustration. This was a fun way for me to try out gganimate.

The approach I took is:

• Get all the Rmd analysis files from the screencast github repo.
• Extract the list of libraries and functions used in each .Rmd file
• Plot frequencies of function use and review functions that I am not aware of

The html file with all the code and results is in this location. The R file used to generate the html file is here.

The plot below shows the how many analyses used a particular package.

The top library as tidyverse is to be expected. It is interesting that lubridate is second. I can see that broom is used quite a bit since after exploratory analysis in the screencast, David explores some models. There are several packages that I was not aware of but I will probably look up the following: widyr, fuzzyjoin, glue, janitor, patchwork and the context in which they were used in the screencast.

The plot below shows the number of functions used from each package.

As expected, most used functions are from ggplot2, dplyr, tidyr since there is lot of exploratory analysis and visualization of data in the screencasts.

The next series of plots shows the individual functions used from the packages.

Based on the above figures, I am listing below some functions that I was not aware of and should learn

• count function in dplyr as a easier way to count for each group or sum a variable for each group.
• geom_col function in ggplot2 for bar graphs
• I became aware of forcats package for working with factors. fct_reorder and fct_lump from the package were used frequently.
• tidyr functions - nest/unnest, crossing, separate_rows
• I realized that I know only a few functions in stringr and should learn more about several functions that were used in the screencast.

I wanted to learn reticulate by trying to create a R version of one of the python notebooks from that class. The class covers the topic of collaborative filtering in lecture 5 and lecture 6. The dataset used is a sample of movielens dataset where about ~670 users have rated ~9000 movies. The objective is to develop a model to predict the rating that a user will give for a particular movie.

The Jupyter notebook for this topic is divided into 2 portions:

• In the first half, the model is developed using just high level fastai functions. The R notebook for the first half is located here.
• In the second half, the model is developed from scratch and 3 different types of models are discussed going from matrix factorization type model to deep learning type models. The R notebook for the second half is located here.

Since the first half involved mainly python functions from fastai library, it seemed like a good use case for reticulate since we could use reticulate just for model development and use R functions for other pre and post processing tasks. The second half involved model building from scratch. In pyTorch, custom models need to be written as python classes. While it was still possible to use reticulate in this case, this may not be the ideal use case since it might be better for somebody developing custom models to do the whole work in python. But once they wrap it into a python package, it is easier to use from R. Overall, reticulate was great to work with and it made it very easy to translate a python function to an equivalent R function. It is a great addition to the R packages.

The general approach that I have followed is:

• Do principal component analysis on the data (each row is a product, each column is a time period (hour of day, day of week or number of days prior to current order))
• Review the loading plots of first two principal components to see purchase patterns
• Identify top 20 products that have high scores in either first or the second principal component
• Check the purchasing pattern by checking the average number of orders for the products that were identified as having top scores in one of the principal components.

Spoiler Alert: Since my analysis is basic, don’t be disappointed if there are no big Aha moments (there will be none). But I think it is still fun to see how we can extract such information directly from data.

I downloaded the data from the following link. The data dictionary is in the following link. The full code with results is in the following location.

Below are some basic info on the datasets

• The number of users are ~200,000.
• The number of orders are ~3.4M. The number of products are ~50K or which ~5K account for 80% of total orders