Role: Data Engineer

Duration: Long-term project.

Location: Redmond WA

Key Skills

Apache Spark (PySpark or Scala) : Writing scalable distributed rule engines.
Apache Hive : Querying partitioned datasets, schema handling.
Amazon EMR : Cluster management, tuning jobs for large datasets (20+ TB).
AWS S3 : Data partitioning, object lifecycle, access patterns.
Glue Data Catalog : Managing Hive table metadata.
Data Quality Frameworks : (Great Expectations, Deequ, or custom rules).
CI/CD : Code deployment pipelines (Git, Jenkins, Airflow, etc