Liang Zhang
San Francisco, California, United States
Summary
🤩
RockstarLiang Zhang is an independent AI researcher and experienced software engineer with 11 years building cloud-native ML infrastructure and distributed systems from San Jose. He shipped production features at Databricks—designing MLflow integrations, a secure MLflow-DBFS filesystem, and performance fixes that resolved complex deadlocks and resource quota issues—and has a track record of delivering under tight deadlines that won key customer deals. His open-source contributions span heavy-hitting projects like Apache Spark, Horovod, Hyperopt, MLflow, and Petastorm, where he improved data type handling, distributed training integrations, and failure-handling for long-running Spark tasks. Comfortable across backend, MLOps, and model lifecycle tooling, he also brings hands-on ML experience in LLMs, interpretability, and reinforcement learning from his independent research. Notably, his internship work vectorized SparkR serialization to achieve ~100x speedups on multi-million row benchmarks—an early sign of his focus on performant data pipelines.
11 years of coding experience
Github Skills (31)
apache-spark10
pytorch10
spark10
python10
testing10
dataframes10
databricks10
big-data10
pyspark10
machine-learning10
mlflow10
dataframe10
scala10
keras10
mlops10
Programming languages (5)
Github contributions (5)
mlflow/mlflow
Nov 2021 - Dec 2022
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
Role in this project:
ML Engineer
ML EngineerContributions:334 reviews, 23 commits, 112 PRs in 1 year 1 month
Contributions summary:Liang primarily contributed to the project by modifying code related to PyTorch and machine learning. They replaced `pytorch_lightning.metrics` with `torchmetrics` across multiple example files and the core PyTorch integration file, streamlining the metric handling within the MLflow framework. Additionally, the user removed and replaced code related to the `try_mlflow_log` function and experimental decorators and made adjustments to functions to improve loading and saving models.
aimlflowmlmodelmachine-learningml
uber/petastorm
Feb 2020 - Apr 2020
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Role in this project:Back-end Developer & DevOps Engineer
Back-end Developer & DevOps EngineerContributions:15 commits, 14 PRs, 3 pushes in 1 month
Contributions summary:Liang primarily contributed to the development and maintenance of the `petastorm` library, focusing on improvements to the Spark dataset converter. Their work included refactoring imports, fixing URI assumptions, and integrating atexit handlers for cleanup. Furthermore, the user implemented features for data type conversion, precision control for floating-point numbers, and support for vector types, enhancing data handling capabilities within the Spark environment. The user also worked on build and release by adding a CI build for pyspark 3.0.
deep-learningdistributed-trainingmachine-learningparquetpetastorm
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.