It complains compile error for command `import pyspark` saying that 'No module named pyspark' in PyCharm provided spark is not installed by `pip install`, whereas it could be imported correctly from python prompt.
Solution:
Find the path for 'SPARK_HOME/python/lib/py4j-0.*.*-src.zip:$SPARK_HOME/python/lib/pyspark.zip'. In PyCharm, open Preferences window, search for 'Project Structure' pane, at the right side, there's a button named 'Add Content Root', add the above two *.zip files here and click OK. Then everything works fine as expected.
It really works. Thanks for this article.
ReplyDeleteAwesomely good and working for me now
ReplyDelete