PySpark Interview Questions and Answers

This image is AI Generated

Conceptual & Theoretical Questions

1. What are the different storage levels supported in Spark? Briefly describe each one
2. When would you choose MEMORY_AND_DISK over MEMORY_ONLY?
3. What is caching in Spark?
4. Why do we use caching and persisting in Spark?
5. When should we avoid using caching?
6. How can you uncache data in Spark?
7. What is the difference between cache and persist?
8. Your Delta table has grown significantly, and query performance is degrading. What steps would you take to optimize it?

Practical Coding Questions

1. Retrieve Customer and Their Orders 50% of Their Highest Order
2. Find Total Orders by Each Customer with List of Ordered Items
3. Find Countries with More Females than Males
4. Find Customers with Big Transactions After 10 Days
5. Behind the Leader: Spot the #2 Salary
6. Identifying Top 3 Performers in Each Department
7. Rank Customers by Lifetime Spending
8. Rank the Employee Salaries and Extract Top Earners
9. Identify Frequent Customers
10. How to Filter and Retrieve the Second Transaction in PySpark
11. How can you find the origin and destination locations for each customer using PySpark
12. How to find the most frequently used word in a text file
13. How to Generate Unique Match Schedule Between Departments
14. How can you unpack the nested list with duplicates and using the list recurssion
15. How can you unpack the nested list with duplicates and with-out using the list recurssion
16. How can you Flatten the list without Duplicates
17. How can you Flatten the list and Count the Occurrences of each item in the list