Save custom transformers in pyspark
When I implement this part of this python code in Azure Databricks: class clustomTransformations(Transformer): <code> custom_transformer = customTransformations() .... pipeline = Pipeline(stages=[custom_transformer, assembler, scaler, rf]) pipeline_model = pipeline.fit(sample_data) pipeline_model.save(<your path>) When I attempt to save the pipeline, I get this: AttributeError: 'customTransformations' object has no attribute '_to_java' Any work arounds? dportman It seems like there is no easy workaround but to try and implement the _to_java method, as is suggested here for StopWordsRemover: