Data Analyst I
Remote
Contracted
Experienced
Job Responsibilities:
- Design, develop, and oversee the implementation of processes and tools to assess the quality of data used in LLM model training.
- Study dataset and quality requirements, and develop automated metrics to measure/monitor data quality at scale.
- Evaluate the annotation data for instruction tuning, preference alignment, and model optimization.
- Collaborate with Data Scientists to scrutinize annotation data and develop strategies for continuous data quality improvement.
- Recommend and implement enhancements to our quality processes, tools, and methodologies based on industry best practices.
Qualifications/Experience:
- Proficiency with applying quantitative methods to structured & unstructured data for complex data analysis, pattern recognition, insights generation, and metrics development.
- Experience with understanding the complexity of user behavior via deep-dive investigations as well as designing, analyzing, and interpreting A/B experiments.
- Strong programming skills in data manipulation & processing (SQL & Python preferred).
- Proven expertise in data wrangling and developing data visualizations & reporting with tooling such as Tableau, AWS, etc.
- Strong understanding of machine learning principles, especially in the context of NLP and LLMs
- 5+ years of design/test/implementation/consulting experience in data quality management for machine learning model training
- Demonstrated experience in project management and cross-functional collaboration.
- Exceptional analytical, problem-solving, and organizational skills
- Strong verbal and written communication skills with the ability to work effectively across internal and external organizations and virtual teams.
- Fluent in Asian languages like Mandarin is a big plus.
Apply for this position
Required*