Setup Configuration

Our lib was developed by use in Microsoft Databricks. So, if you need to use in your Databricks, the steps bellow is necessary to start.

Creating an init script

First, you have to create an init script to include our lib in your cluster initialization. There are two ways to create an init script in your Databricks environment, but we recommend a global init script. You will have to create only once and all the clusters in a same Databricks will be able to use Pyiris.

The init script

pip install git+https://$PYIRIS_TOKEN:x-oauth-basic@github.com/cervejaria-ambev/pyiris.git@vX.X.X

OBS: replace X.X.X to the current pyiris version number.

To create the global init script, access the Admin Console -> Global Init Script and click in the button + Add. In the name, write “install-pyiris” and paste the script above.

The environment variables

Also, you have to set some environment variables to perform a good work:

KEY_VAULT_NAME=kv-nonprod-iris or kv-prod-iris
ENVIRONMENT_PERMISSION=PUBLIC or PRIVATE
ENVIRONMENT=dev or prod
PYIRIS_SECRET={{secrets/KeyVault//IrisLakeSecret}}
PYIRIS_TENANT_ID={{secrets/KeyVault//IrisLakeTenantID}}
PYIRIS_CLIENT_ID={{secrets/KeyVault//IrisLakeClientID}}
PYIRIS_TOKEN={{secrets/KeyVault//pyirisGit}}

OBS: you have to guarantee that your Databricks is connected with IRIS Key-Vault, accord to your environment. Also, have to get your correct Key Vault scope.

OBS 2: the “ENVIRONMENT_PERMISSION” variable have to be accord to your environment permission: PUBLIC or PRIVATE; as ENVIRONMENT variable, has to be “dev”or “prod”.

To insert the environment variables access, individual, your clusters in Clusters -> select one cluster -> Edit -> Advanced Options -> insert the above environment variables in the Environment Variables text box.

Good job, the Pyiris was successfully installed! :)