Data Engineer (from Media and Entertainment Industry Background)

---

Job description

Job Title: Data Engineer (from Media and Entertainment Industry Background)

Location: 100% Remote
Duration: 10+ Months

No. of Positions: 2

Job Summary:
We are seeking a highly skilled Senior Data Engineer to join our dynamic technology team. The ideal candidate will have a strong background in data engineering, data architecture, and cloud-based data solutions, particularly with AWS services. This role is critical for building and maintaining robust data pipelines, ensuring data quality, and enabling data-driven decision-making across the organization.

Key Responsibilities:
Data Pipeline Development: Design, implement, and maintain scalable data pipelines to process large volumes of data using AWS services, ensuring high availability and performance.
Data Processing: Utilize Apache Spark, Databricks, or Snowflake to transform and analyze data from various sources, supporting business analytics and reporting needs.
Data Modeling: Use DBT for data transformation, ensuring that data models are accurate, efficient, and maintainable.
Collaboration: Work closely with data analysts, data scientists, and business stakeholders to understand data needs and deliver data solutions that meet those requirements.
Data Quality Assurance: Monitor data quality metrics, identify discrepancies, and implement corrective measures to maintain data integrity and accuracy.
Documentation: Create and maintain detailed documentation of data architectures, pipelines, and workflows for future reference and knowledge transfer.
Optimization: Continuously evaluate and optimize data storage, processing, and retrieval strategies for efficiency and scalability.
Troubleshooting: Investigate and resolve data issues in a timely manner, providing support for data-related queries and challenges.
Best Practices: Promote best practices in data engineering, including coding standards, testing methodologies, and data governance.

Must-Have Skills:
AWS Services: Strong experience with AWS data services (e.g., S3, Redshift, Glue, Lambda) for data storage, processing, and orchestration.
SQL: Proficient in writing complex SQL queries for data extraction, manipulation, and analysis.
Python: Solid programming skills in Python for data processing, automation, and API integration.
Apache Spark: Hands-on experience with Apache Spark for distributed data processing and analytics.
Data Lakes: Familiarity with Databricks or Snowflake as data warehousing solutions to support analytical workloads.
DBT: Proficiency in using DBT for data transformation and building modular, testable data models.

Soft Skills:
Excellent Communication: Ability to convey complex technical concepts to non-technical stakeholders and work collaboratively within cross-functional teams.
Adaptability: Comfortable working in a fast-paced, evolving environment and adapting to changing priorities and technologies.
Collaboration: Strong teamwork and interpersonal skills to foster a collaborative work environment.
Analytical Skills: Exceptional analytical and problem-solving skills with a focus on detail, consistency, and quality.
Critical Thinking: Ability to think critically and strategically about data architecture and engineering challenges.

Nice-to-Have Skills:
Kafka: Familiarity with Apache Kafka for real-time data streaming and processing.
CI/CD: Experience with Continuous Integration and Continuous Deployment tools and practices to automate the deployment of data solutions.
Airflow: Knowledge of Apache Airflow for orchestrating and scheduling data workflows and pipelines.
Fivetran: Understanding of Fivetran for seamless data integration from various sources to data warehouses.

Qualifications:
Bachelors degree in Computer Science, Engineering, Information Technology, or a related field.
Specify years of experience, e.g., 10+ Years in data engineering, data architecture, or a related field.
Experience working with cloud-based data solutions and big data technologies.

<!– job description page –>