Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 19054

VSCode: pyspark runs fine when using "run python file" option. But throws error when using "run code" option

$
0
0
from pyspark.sql import SparkSessionspark = SparkSession.builder.appName("Datacamp Pyspark Tutorial").config("spark.memory.offHeap.enabled","true").config("spark.memory.offHeap.size","10g").getOrCreate()df = spark.read.csv('datacamp_ecommerce.csv',header=True,escape="\"")df.show(5,0)

enter image description hereWhen I run using the option Run Python file (shown in the image above), it successfully displays the output in the TERMINAL tab of VSCode.

But when I use the first option Run Code (shown in image above), it throws the following error in the OUTPUT tab of VSCode.

[Running] python -u "c:\VSCode_PyProjects\VSCode_PysparkProj\venv\test.py"Traceback (most recent call last):File "c:\VSCode_PyProjects\VSCode_PysparkProj\venv\test.py", line 2, in from pyspark.sql import SparkSessionModuleNotFoundError: No module named 'pyspark'

Remarks:

  1. As shown in the image below, the env folder contains pyspark.
  2. I have python, spark, pyspark, and java installed on Windows-10 using this post.
  3. I've also installed popular Code Runner extension in VSCode.

Question: Why the code works fine with Run Python File, but not with Run Code option. What I may be missing here, and how we can fix the issue?

enter image description here


Viewing all articles
Browse latest Browse all 19054

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>