Lead Data Engineer Job at Argyllinfotech, San Jose, CA

M3B4K2xZNG5pNnBmT3dDY0pURTJFUkVncnc9PQ==
  • Argyllinfotech
  • San Jose, CA

Job Description

Hello Everyone,

No GC,

Role: Senior Lead Data Engineer

Location: San Jose, CA 95125 Hybrid(Locals)

Key: Hadoop, Spark SQL strong

Key Responsibilities:

Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL

- Manage and optimize the infrastructure ensuring reliability and performance

-Design and implement data models and schemas for enterprise data warehouse and data marts

-Write and maintain scripts in Python for various applications

-Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases

- Work closely with analysts and product managers to understand data requirements and deliver high-quality data

-Provide technical guidance and mentorship to other team members fostering best practices in data engineering

-Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability

Required Skills:

-High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing

-Good knowledge in Python for data manipulation and scripting

-Mastery of advanced SQL and deep experience working with relational databases and data warehouses

-Hands-on experience with workflow management tools like Apache Airflow

-Familiarity with Unix shell scripting for automation and system management

Job Tags

Local area,

Similar Jobs

S4L Partners

Bilingual CPA Job at S4L Partners

 ...This is a remote position. Remote Bilingual Tax CPA Salary:$120,000.00 Schedule: MondayFriday, 8:00 AM 5:00 PM ET Were looking for a Bilingual Tax CPA to lead our U.S. tax operations and collaborate closely with our Puerto Rico tax... 

Power3 Solutions and Partnering Companies

Test Engineer L2 Job at Power3 Solutions and Partnering Companies

We are looking for Test Engineers to join their rapidly growing team! Position Description Collaborate with cross-functional agile teams to understand project requirements and user stories. Understanding HW Intricacies to help prioritize testing based on risks... 

Gourmet Foods International

Class A Company Driver Job Job at Gourmet Foods International

Class A Company Driver JobRESPONSIBILITIES AND DUTIES Performing all pre-trip responsibilities including: checking route numbers and account numbers for assigned deliveries, counting items and checking customer invoices of products that have been loaded, and performing... 

Selective Investigations Inc

Bookkeeper Part-Time Job at Selective Investigations Inc

 ...Investigations, Inc. is seeking a reliable part-time experienced bookkeeper (employee or 1099) to handle accounts receivables/payables, and...  ...access to internet Have own computer equipment Benefits: Approximately 12 - 16 hours per week This is a remote position.

Lincoln Financial

Actuary, Life Inforce Model Development Job at Lincoln Financial

 ...assistance:is not available for this opportunity. Requisition #: 74686 The Role at a Glance As an Actuary of the Life Inforce Model Development team, you will be involved in various model development projects to enhance and refine the inforce model that is...