View all jobs

Data Pipeline Engineer

Vancouver, BC · Information Technology
On behalf of our global enterprise client, Affinity is currently seeking an experienced Data Pipeline Engineer to develop as well as providing maintenance, administration and technical support for data pipelines between operational data sources and Google Cloud.    


● Design, build and deploy Data Pipelines on Google Cloud for Operational Data including Historian time series data, SQL server databases and unstructured data sources.
● Solve complex technical problems with legacy systems to overcome technical roadblocks preventing data migration to the cloud platform
● Perform pipeline data validation and ensure that data delivery, data security and data quality are not compromised
● Provide and develop monitoring and performance metrics for data-pipelines to measure and ensure performance KPIs are met
● Maintain application and system documentation
● Facilitate and support incident response and forensics for pipelines on Google Cloud Platform  


● Bachelor's Degree in Computer Science, Computer Engineering or a related discipline, and 5 years of proven experience in the ETL field
● Hands-on experience with Google Cloud Platform, relational databases and ETL methodology, strong scripting skills 
● Relational and non-relational data stores design and deployment, including data retention and archiving strategy
● Data processing management and development, including real-time data ingestion and batch processing
● Data solutions monitoring and optimization for BLOB, Data Lake and relational stores
● Data processing solutions design and implementation, including ingestion method optimization, processing capacity allocation 
● Design data solutions for data security and compliance, including endpoints security, use of reliable authentication methods, data encryption, data masking, auditing and data classification. 
● Strong aptitude for learning new technologies and analytics techniques
● Demonstrated problem solving ability
● Familiarity with programming technologies and languages including Java, T-SQL, .NET C# and Visual Basic, Python, SSIS, CDAP APIs.
● Knowledge of general computer controls related to operations, information security and change management of systems software, application source code, network, and system database technologies
● Strong cross-functional team participant and collaborative approach to problem solving
About Affinity Staffing:
Affinity Staffing is a full service Information Technology agency that takes a unique approach to recruiting. We believe recruiting is about creating long term relationships that foster a mutually beneficial partnership - an affinity. Bringing a new style of recruiting founded on four core principles – Transparency – Flexibility – Efficiency – Agility.
For more information on Affinity Staffing, please visit.
Powered by