Data and Cloud Freelancer
There are several ways to query delta tables from PowerBi. We are going to cover the 4th method here. To do it first we need a service princpal, a secret scope pointing to a databricks keyvault and the password of the SPN stored in this keyvault. Once we have this, the first step is to …
I’ve started working on some of my free time in a project to build the smallest useful analytics platform on the cloud (starting with azure). The purpose is to use it a sa PoC to show to colleagues, managers, prospective customers or just to have fun and play It’s publicly available on my github repo …
There are many ways to implement some CI/CD with Databricks. We can use Azure DevOps, Github+Github Actions or any other combination of tools, including the dbx tool. But an easy way to just copy notebooks between workspaces can be implemented easily with Azure DevOps. We are going to use the git repos capability of Azure …
In this article i will show how to avoing going outside to the internet when using resources inside azure, specially if they are in the same subscription and location (datacenter). Why we may want a private endpoint? Thats a good question. For oth security and performance. Just like using TSCM Equipment for optimal safety and …
Most of the developments I see inside databricks rely on fetching or writing data to some sort of Database. Usually the preferred method for this is though the use of jdbc driver, as most databases offer some sort of jdbc driver. In some cases, though, its also possible to use some spark optimized driver. This …
In todays post I am going to share an ansible playbook to configure a new VM recently launched. This playbook contains the following: Change Admin password Create linux Group Add user to several groups Create a new user with specific salted password (Check point 3 for generating the hashed salt) Find all the mounted disks …
In this article we will see how to provision an azure synapse cluster, load some large quantity of data from azure blob storage and run a query to see the contents and check performance. I plan to write a serie of articles arround data warehousing in the cloud so check out for new articles soon. …
In this post we will deploy the azure infrastructure to have an storage account queue. For this we will use terraform which can be downloaded from here. Before start, we need a few prerequisites, these are the following: Installing terraform, azure-cli and azure-storage-queue Create a service principal for deploying the resources with terraform Creating the …