Dice is the leading career destination for tech experts at every stage of their careers. Our client, Info Dinamica Inc, is seeking the following. Apply via Dice today!
Role: Senior Data Engineer with Databricks
Remote (US) with ability to cover a few PST hours
Job Type: W2 Contract
Experience Level: Senior
Primary Skills:
Databricks, Spark/PySpark, Kafka, Delta Lake, Python, SQL, PostgreSQL, AWS, Datadog, PagerDuty
Key Responsibilities:
• Respond promptly to PagerDuty and Datadog alerts, triaging incidents efficiently.
• Debug and resolve failures across streaming and batch data pipelines.
• Troubleshoot Databricks/Spark job failures, Kafka lag/connectivity issues, Delta Lake checkpoint failures, PostgreSQL sink issues, and schema-related failures.
• Restart jobs, apply configuration fixes, and escalate issues with detailed root cause analysis as needed.
• Execute operational runbooks and maintain thorough incident documentation and postmortems.
• Improve platform stability by reducing recurring incidents and alert noise.
• Coordinate effectively with downstream consumers and engineering teams during incidents.
Required Skills:
• Strong hands-on experience with Databricks, Spark/PySpark, Kafka, Delta Lake, Python, SQL, and PostgreSQL.
• Experience with AWS services including S3, IAM, Secrets Manager, and KMS.
• Knowledge of Datadog, PagerDuty, and production observability practices.
• Strong troubleshooting and debugging skills across distributed systems and streaming pipelines.
• Ability to distinguish between transient infrastructure issues, configuration fixes, and application/code defects.
Preferred Skills:
• Experience with Apache Flink and stateful streaming systems.
• Healthcare or HIPAA domain exposure.
• Experience with dbt, Great Expectations, or data quality frameworks.
• Terraform-managed cloud infrastructure experience.
Apply tot his job
Apply To this Job