Testing pyspark with pytest
Posted on Sun 15 January 2023 in Python • Tagged with spark, pyspark, pytest, testing
Using pytest to test your pyspark code
In my past couple of roles I have been developing on the pyspark platform and needed to to add tests to the code as part of the CI pipeline. I couldn't really find much out there that describes the process neatly, mostly a …
Continue reading