Blog

Smallest Analytical Platform Ever!

I’ve started working on some of my free time in a project to build the smallest useful analytics platform on the cloud (starting with azure). The purpose is to use it a sa PoC to show to colleagues, managers, prospective customers or just to have fun and play It’s publicly available on my github repo …

Centos 8/9 Stream and AlmaLinux images for WSL

For the ones like me, interested in running Linux systems on windows for many automation or administration tasks, I am sharing here the images i’ve found: Centos7/8/9 Stream: https://github.com/mishamosher/CentOS-WSL AlmaLinux (Centos Replacement equivalent to RHEL): https://www.microsoft.com/en-us/p/almalinux-8-wsl/9nmd96xjj19f#activetab=pivot:overviewtab The latest one is a direct link to the microsoft store.

Databricks cluster policies at a glance. The easy way!

For these administering one or more databricks workspaces, cluster policies are an important tool where we spend some time with. Introduction But what are cluster policies? Cluster policies are basically a json file with some parameters that we use to allow (or not) users to select certain things when creating a cluster. Not only users …

Using Azure Private Endpoints with Databricks

In this article i will show how to avoing going outside to the internet when using resources inside azure, specially if they are in the same subscription and location (datacenter). Why we may want a private endpoint? Thats a good question. For oth security and performance. We dont want the traffic going outside to the …

Databricks connectivity to Azure SQL / SQL Server

Most of the developments I see inside databricks rely on fetching or writing data to some sort of Database. Usually the preferred method for this is though the use of jdbc driver, as most databases offer some sort of jdbc driver. In some cases, though, its also possible to use some spark optimized driver. This …

Ansible playbook to configure Azure Red Hat VM’s

In todays post I am going to share an ansible playbook to configure a new VM recently launched. This playbook contains the following: Change Admin password Create linux Group Add user to several groups Create a new user with specific salted password (Check point 3 for generating the hashed salt) Find all the mounted disks …

Databricks and Spark Crash Course. Delta and More!

I’ve been working on a Databricks and Delta tutorial for all of you. I published it as notebook and you can grab it here. We will load some sample data from the NYC taxi dataset available in databricks, load them and store them as table. We will use then python to do some manipulation (Extract …

Unload data from AWS Redshift to S3 in Parquet

Following the previous redshift articles in this one I will explain how to export data from redshift to parquet in s3. This can be interesting when we want to archive (infrequently queried) data to be queried cheaper with spectrum, or to store in s3 archive, or to export to another storage solution like glacier. The …