Wei Liu
Member Of Technical Staff at xAI
New York, New York, United States
Summary
🤩
Rockstar🎓
Top SchoolWei Liu is a software engineer with 8 years of experience building scalable data systems, currently working on pretraining data at xAI after roles at Netflix and Databricks focused on Spark, structured streaming, and Spark Connect. A frequent contributor to Apache Spark, he has improved testing infrastructure and stability for core streaming features—work that directly supports large-scale, production-grade analytics. He holds an MS in Computer Science from the University of Chicago and a BS in Math & CS from UIUC, with an exchange stint at the University of Tokyo, reflecting strong theoretical and international training. Combining back-end systems expertise with test automation rigor, he brings a pragmatic focus on reliability and reproducibility to high-throughput ML and data platforms.
8 years of coding experience
4 years of employment as a software developer
University of TokyoHigh School Diploma, High School Diploma at Beijing 101 Middle SchoolBachelor of Science - BS, Mathematics and Computer Science, Bachelor of Science - BS, Mathematics and Computer Science at University of Illinois Urbana-ChampaignMaster of Science - MS, Computer Science, Master of Science - MS, Computer Science at University of Chicago
Chinese, English, Japanese
Github Skills (11)
testing10
stream10
spark10
spark-structured-streaming10
streaming10
scala10
java9
javas9
data-engineering8
sql8
big-data7
Programming languages (6)
Github contributions (5)
apache/spark
Sep 2022 - Oct 2025
Apache Spark - A unified analytics engine for large-scale data processing
Role in this project:
Back-end Developer & Test Automation Engineer
Back-end Developer & Test Automation EngineerContributions:364 reviews, 153 PRs, 515 comments in 3 years 1 month
Contributions summary:Wei primarily focused on improving the testing infrastructure and implementing new tests within the Apache Spark codebase. Their contributions involved adding and enhancing test suites, specifically targeting functionality related to streaming session windows, stateful operators, and query execution errors. The user also addressed code quality issues and ensured compatibility by fixing typos and making code style improvements. The user's work directly supports the testing and stability of core structured streaming features.
apache-sparkpythonscalarjava
WweiL/oss-spark
Sep 2022 - Feb 2026
Apache Spark - A unified analytics engine for large-scale data processing
Contributions:4 PRs, 540 pushes, 132 branches in 3 years 5 months
analyticsdata-processingapachebig-dataspark
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.