Company: General Motors Financial
Job description: Job Category: Information TechnologyJob Description:Why General Motors Insurance?At General Motors Insurance, we are building an Insurtech business that will reinvent auto insurance. We are fully owned and backed by auto industry leaders General Motors and GM Financial. This is a truly unique opportunity to join at the foundational stage of a start-up leading the transformation of the auto insurance experience.GM has the largest connected vehicle fleet worldwide. In the US alone, there are currently 9M+ connected GM vehicles on the road and that number is projected to triple in the next 10 years. More than that, the OnStar system currently has access to over 900 data points from the vehicle. This surge in information about vehicles and how they are driven will revolutionize auto insurance. This disruption is backed by the bold GM vision of zero crashes, zero emissions and zero congestion. We are serious about the safety and financial security of our customers. If you are passionate about driving innovation and delivering results in a fast-paced, value-focused environment, General Motors Insurance is looking for you.Position open until filled.Note: We are unable to offer sponsorship for this position.Responsibilities:About the role:We are expanding our efforts into complementary data technologies for decision support in areas of ingesting and processing large data sets, such as vehicle telematics data, including data commonly referred to as semi-structured or unstructured data. Our interests are in enabling data science and search-based applications on large and low latent data sets in both a batch and streaming context for processing. To that end, this role will engage with team counterparts in exploring and deploying technologies for creating data sets using a combination of batch and streaming transformation processes. These data sets support both off-line and in-line machine learning training and model execution. Other data sets support search engine-based analytics. Exploration and deployment of technologies activities include identifying opportunities that impact business strategy, collaborating on the selection of data solutions software, and contributing to the identification of hardware requirements based on business requirements. Responsibility also includes coding, testing, and documentation of new or modified scalable analytic data systems including automation for deployment and monitoring. This role participates along with team counterparts to develop solutions in an end-to-end framework on a group of core data technologies. Other aspects of the role include developing standards and processes for data engineering projects and cloud initiatives.JOB DUTIES
- Code, test, deploy, Orchestrate, monitor, document and troubleshoot cloud-based data engineering processing and associated automation in accordance with best practices and security standards throughout the development lifecycle.
- Work closely with data scientists, data architects, ETL developers, other IT counterparts, and business partners to identify, capture, collect, and format data from the external sources, internal systems and the data warehouse to extract features of interest.
- Significantly contribute to the evaluation, research, experimentation efforts with batch and streaming data engineering technologies to keep pace with industry innovation while assessing business impact and viability for use cases associated with efforts in hand.
- Work with data engineering related groups to inform on and showcase capabilities of emerging technologies and to enable the adoption of these new technologies and associated techniques.
- Significantly contribute to the definition and refinement of processes and procedures for the data engineering practice.
- Educate and develop ETL developers on data engineering cloud-bases initiatives so as to enable transition to data engineer and practice.
- Develop and maintain data quality frameworks to ensure data accuracy, completeness, and reliability across all data platforms.
- Build and maintain CI/CD pipelines for automated testing and deployment of data engineering solutions.
- Optimize data systems performance, scalability, and reliability through monitoring and tuning.
- Implement data observability and monitoring solutions to ensure data pipeline reliability and performance.
- Experience with processing large data sets such as Telematics data using Databricks, Hadoop, HDFS, Spark/PySpark, Kafka, Flume or similar distributed systems and modern streaming architectures.
- Experience with ingesting various source data formats such as JSON, Parquet, Avro, CSV, XML, YAML, SequenceFile, Cloud Databases, MQ, Relational Databases such as Oracle, SQL Server.
- Experience with Cloud technologies (such as Azure, AWS, GCP) and native toolsets such as Azure ARM Templates, Hashicorp Terraform, AWS Cloud Formation.
- Understanding of cloud computing technologies, business drivers and emerging computing trends.
- Thorough understanding of Hybrid Cloud Computing: virtualization technologies, Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS) Cloud delivery models and the current competitive landscape.
- Working knowledge of Object Storage technologies to include but not limited to Azure Data Lake Storage Gen2, S3, Minio, Ceph, ADLS etc.
- Experience with containerization to include but not limited to Dockers, Kubernetes, Spark on Kubernetes, Spark Operator.
- Working knowledge of Agile development /SAFe, Scrum and Application Lifecycle Management.
- Strong background with source control management systems (GIT or Subversion); Build Systems (Maven, Gradle, Webpack); Code Quality (SonarQube); Artifact Repository Managers (Artifactory), Continuous Integration/ Continuous Deployment (CI/CD and Azure DevOps), Identity & Access Management (Azure Active Directory, Key Vault, Entra ID, Databricks Unity Catalog) and data quality frameworks.
- Experience with NoSQL data stores such as Azure CosmosDB, MongoDB, Cassandra, Redis, Riak or other technologies that embed NoSQL with search such as MarkLogic or Lily Enterprise.
- Creating and maintaining ETL processes.
- Knowledgeable of best practices in information technology governance and privacy compliance.
- Experience with Adobe solutions (ideally Adobe Experience Platform, DTM/Launch) and REST APIs.
- Troubleshoot complex problems and works across teams to meet commitments
- Excellent computer skills and proficiency in digital data collection
- Ability to work in an Agile/Scrum team environment
- Strong interpersonal, verbal, and writing skills
- Digital technology solutions (DMPs, CDPs, Tag Management Platforms, Cross-Device Tracking, SDKs, etc)
- Knowledge of Real Time-CDP and Journey Analytics solutions
- Understanding of big data platforms and architectures, data stream processing pipeline/platform, data lake and data lake houses
- SQL experience: querying data and sharing what insights can be derived
- Understanding of cloud solutions such as Google Cloud Platform, Microsoft Azure & Amazon AWS cloud architecture & services
- Understanding of GDPR, privacy & security topics
- 6-8 years of hands-on experience with data engineering required
- Bachelor's Degree in related field or equivalent work experience required
Expected salary:
Location: Arlington, TX
Apply for the job now! Apply Now