# Data Transform The transformation module is responsible for making the data transformations. ### SQL transformation This module makes SQL expressions available to transform. Example: ~~~Python from pyiris.ingestion.transform import SqlTransformation sql_transformation = SqlTransformation(name='divide', description='Unit price division', to_column="unit_price", sql_expression="price/quantity") transformed_dataframe = sql_transformation.transform(dataframe=extracted_dataset) ~~~ ### Hash transformation This module returns a hash transformation based on an inputted column. Example: ~~~Python from pyiris.ingestion.transform import HashTransformation hash_transformation = HashTransformation(name='Hash CPF', description='Hash CPF to be according to LGPD', from_columns=["cpf"]) transformed_dataframe = hash_transformation.transform(dataframe=extracted_dataset) ~~~ ### Custom transformation This module gives for the user tools to customize the dataframe, with the main custom features. Example of uses: ~~~Python from pyiris.ingestion.transform.transformations.custom.custom import divide from pyiris.ingestion.transform.transformations.custom_transformation import CustomTransformation custom_transformation = CustomTransformation(name='middle_price', description='Dividing two fictitious columns (price/quantity) to generate column middle_price', method=divide, to_column='middle_price', column1='price', column2='quantity') transformed_dataframe = custom_transformation.transform(dataframe=extracted_dataset) ~~~ #### Custom transformation - snakecase_column_names This method intends to rename all columns of a given dataframe to snake case. The transformations applied are: - replacing letters containing accents and special characters (e.g. replacing 'á', 'à', 'ã' or 'â' to 'a'); - replacing uppercase letters with underscore and lowercase; - removing duplicated undescore; - removing leading and trailing undescore; - removing all characters that are not allowed (a-z0-9_) ###### Example code ~~~Python from pyiris.ingestion.transform.transformations.custom.custom import snakecase_column_names from pyiris.ingestion.transform.transformations.custom_transformation import CustomTransformation custom_transformation = CustomTransformation(name="snakecase_column_names", description="rename columns to snake case", method=snakecase_column_names) transformed_dataframe = custom_transformation.transform(dataframe=extracted_dataset) ~~~ ###### Example outputs already_snake_case_column_name → already_snake_case_column_name notSNAKECaseColumnNameOne → not_snake_case_column_name_one NÕTSnákêCãsèColùmnNãmêTWÕ → not_snake_case_column_name_two ## Transform Service The class **pyiris.ingestion.transform.TransformService** works as a service. You can execute some transformations in sequence, or only one. Follow the example of uses: ~~~Python from pyiris.ingestion.transform import TransformService, HashTransformation, SqlTransformation transform_service = TransformService( transformations=[ SqlTransformation(name='divide', description='Getting middle price', to_column="middle_price", sql_expression="price/quantity"), HashTransformation(name='Hash CPF', description='Hash CPF to be according to LGPD', from_columns=["seller_cpf"]) ] ) transformed_dataframe = transform_service.handler(dataframe=extracted_dataset) ~~~ To have more information, please, access the code docstring in **Pyiris modules**.