Example of pyspark spline atlas

[cometta]

Any example of running pyspark job with spline jars files and push integrate with apache atlas? I can only find example in scala

[wajda]

Atlas integration is out of direct scope for the Spline team. There are a number of examples on the internet, some of them are listed here - https://absaoss.github.io/spline/articles.html
Make sure you are referring to the more recent examples though, the ones that involve Spline version 0.5+
Regarding Scala vs Python, there is no real difference with respect to integration with Atlas. If you can make Spline working for your Python job, the rest steps are pretty much the same. The principle is simple - the Spline agent collects the meta data from your job and sends it over to the destination you like. The destination is specified by the "LineageDispatcher" in use. See https://github.com/AbsaOSS/spline-spark-agent#dispatchers