- Design, develop, and maintain data pipelines to extract, transform, and load (ETL) data from various sources into our data lake on Amazon S3.
- Implement data validation and quality checks to ensure the accuracy and consistency of data throughout the ETL process.
- Utilize Amazon Athena for ad-hoc querying and analysis of data stored in Amazon S3, optimizing query performance and efficiency.
- Develop and maintain automated workflows using Apache Airflow to schedule and orchestrate data processing tasks, ensuring timely and reliable data delivery.
- Write clean and efficient Python code to support data transformation, integration, and data engineering tasks.
- Collaborate with senior data engineers to design and evolve the data architecture, ensuring scalability and performance of the data infrastructure.
- Maintain comprehensive documentation of data pipelines, workflows, and processes for future reference.
- Identify and resolve data pipeline issues, performance bottlenecks, and data quality concerns in a timely manner.
- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide data engineering support.
- Display exemplary conduct and live by the Group's Values and Code of Conduct.
- Take personal responsibility for embedding the highest standards of ethics, including regulatory and business conduct, across Standard Chartered Bank. This includes understanding and ensuring compliance with, in letter and spirit, all applicable laws, regulations, guidelines and the Group Code of Conduct.
- Effectively and collaboratively identify, escalate, mitigate and resolve risk, conduct and compliance matters.
- Lead the team to achieve the outcomes set out in the Bank's Conduct Principles: [Fair Outcomes for Clients; Effective Financial Markets; Financial Crime Compliance; The Right Environment.] *
- Exercise authorities delegated by the Board of Directors and act in accordance with Articles of Association.
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- Strong knowledge of Amazon S3, Amazon Athena, and Apache Airflow.
- Proficiency in Python programming for data manipulation and scripting.
- Basic understanding of data modelling, SQL and database concepts.
- Excellent problem-solving and troubleshooting skills.
- Strong communication and teamwork abilities.
- Eagerness to learn and adapt to new technologies and tools.
- Experience with cloud platforms like AWS.
- Familiarity with distributed computing frameworks ( Apache Spark).
- Knowledge of data warehousing concepts.
- Previous experience with data visualization tools ( Tableau, Power BI).
- Previous experience with ML.
- Previous experience with Terraform.
- Data Pipeline Development
- Data Quality Assurance
-
TEKsystems Guangzhou, 中国Job Summary · 工作生活平衡 · 行业顶尖外企 · 技术革新,提升空间大 · 公司介绍: · 行业顶尖外资银行,具有独立的IT研发团队,IT技术人员将近30000人 · 岗位职责: · 1、负责银行数据开发项目,主要包括数据接入、数据存储、数据治理、数据服务、数据分析等; · 2、参与大数据平台的建设与维护,包括数据集成、数据模型设计、数据质量、数据安全、性能优化等; · 3、对业务有很好的理解,能从业务角度解读业务需求,并转化为程序开发任务; · 4、与业务团队保持良好沟通,了解业务需求,提供高质量的技术支持; · 5、参与大数据开发团队 ...
-
TEKsystems Guangzhou, 中国职位介绍 · 工作生活平衡 · 行业顶尖外企 · 技术革新,提升空间大 · 公司介绍: · 行业顶尖外资银行,具有独立的IT研发团队,IT技术人员将近30000人 · 岗位职责: · 1、负责银行数据开发项目,主要包括数据接入、数据存储、数据治理、数据服务、数据分析等; · 2、参与大数据平台的建设与维护,包括数据集成、数据模型设计、数据质量、数据安全、性能优化等; · 3、对业务有很好的理解,能从业务角度解读业务需求,并转化为程序开发任务; · 4、与业务团队保持良好沟通,了解业务需求,提供高质量的技术支持; · 5、参与大数据开发团队的日常管理,包 ...
-
Standard Chartered Bank Guangzhou, 中国 全职Role Responsibilities · We're looking for a Data Platform Engineer to work onsite within our brand new Digital Bank. We're small right now, but a growing venture, with new and exciting problems to solve. We work in project-based sprints in small, interdisciplinary teams. · As a ...
-
Artefact Huangpu, 中国What you will be doing: · Key responsibilities · As a Data Engineer Intern, your role will encompass: · ● Supporting to conduct ambitious projects in the transformation of clients through data · ● Collaborating with the consulting and data team to provide comprehensive services t ...
-
微信表情搜索算法工程师
3天前
Tencent Guangzhou, 中国微信表情搜索算法工程师 广州 分享 分享岗位 · 方式1:复制岗位链接 · 方式2:分享岗位海报 · 手机扫描二维码分享 · 收藏 WXG 点击了解更多BG信息 技术 两年以上工作经验 更新于年05月17日 岗位职责 1.参与微信搜一搜表情搜索方向算法优化;2.通过技术优化,不断提升产品效果,提升产品使用率;3.工作内容包括并不限于case分析、数据挖掘、特征工程、召回策略、排序策略、相关性等。 · 岗位要求 1.硕士及以上学历,3年或以上工作经验;2.有离线大数据处理、在线c++开发、深度模型开发经验;3.对NLP、数据挖掘、深度学习一 ...
-
Tencent Guangzhou, 中国微信视频号高级/专家算法工程师(直播带货) 广州 分享 分享岗位 · 方式1:复制岗位链接 · 方式2:分享岗位海报 · 手机扫描二维码分享 · 收藏 WXG 点击了解更多BG信息 技术 三年以上工作经验 更新于年05月20日 岗位职责 1.负责视频号直播带货内容自然推荐,进行个性化分发;2.负责策略、精排、粗排、召回等链路优化;3.通过数据分析追踪效果,推动整体效果的优化。 · 岗位要求 1.本科及以上学历,硕士、博士优先,3年以上相关工作经验;2.有视频流电商相关经验优先;3.深入掌握机器学习、深度学习算法及原理;4.优秀的分析问题、 ...
-
Tencent Guangzhou, 中国微信读书 iOS/Android 客户端开发工程师 广州 分享 分享岗位 · 方式1:复制岗位链接 · 方式2:分享岗位海报 · 手机扫描二维码分享 · 收藏 WXG 点击了解更多BG信息 技术 一年以上工作经验 更新于年05月15日 岗位职责 1.负责微信读书iOS/Android客户端的开发维护工作,包括性能调优、技术难点攻坚等;2.参加专项技术调研、新技术探索;3.参加性能和稳定性相关的体系建设、相关的工具开发,保证软件质量和研发效率。 · 岗位要求 1.计算机相关专业本科及以上学历,一年以上客户端应用开发经验;2.熟悉 Objec ...
-
TEKsystems Guangzhou, 中国Job Summary · AWS Cloud 或者其他Cloud 也可以 · Python Airflow · Kafka English needs to be fluent, prefer senior candidates · Job Description: · We are seeking a highly motivated Data Engineer to join our dynamic Data Engineering team. As a Data Engineer, you will play a crucial role i ...
-
HSBC Guangzhou, 中国 Fixed Term - 全职Some careers have more impact than others. · If you're looking for a career where you can make a real impression, join HSBC and discover how valued you'll be. · We are currently seeking an experienced professional to join our team in the role of Software Engineer Manager. · Bu ...
-
CSA Group Guangzhou Shi, 中国 全职Employment Status: · Regular Time Type: · Full time BUILDING A WORLD CLASS TEAM STARTS WITH YOU · At the heart of CSA Group is a vision: making the world a better, safer, more sustainable place. It's been part of our mission for nearly one hundred years: from the first e ...
-
Arup Guangzhou, 中国To expand our business further, we are now looking for highly motivated candidates to fill a variety of positions. Choose Arup China to launch your career and grow together Graduate Structural Engineer · Responsibility: · To assist senior engineer with structural member design of ...
-
MS Engineer
3天前
NTT DATA Guangzhou, 中国Job Description · NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. · In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented ...
-
HSBC Guangzhou, 中国 Fixed Term - 全职Some careers have more impact than others. · If you're looking for a career where you can make a real impression, join HSBC and discover how valued you'll be. · We are currently seeking an experienced professional to join our team in the role of Head of Connect Engineering · B ...
-
Manager, QA
1周前
Thermo Fisher Scientific Guangdong, 中国 全职Job Description · Position Summary: · Lead a team of quality engineers/inspectors to manage and improve quality for Thermo China manufacture to achieve defined quality and meet production needs through initial supplier quality system and procedures assessment, managing on going ...
-
Manager, QA
1周前
ThermoFisher Scientific Guangdong, 中国 Paid WorkJob Description · Position Summary: · Lead a team of quality engineers/inspectors to manage and improve quality for Thermo China manufacture to achieve defined quality and meet production needs through initial supplier quality system and procedures assessment, managing on going p ...
-
HSBC Guangzhou, 中国 Fixed Term - 全职Some careers have more impact than others. · If you're looking for a career where you can make a real impression, join HSBC and discover how valued you'll be. · We are currently seeking an experienced professional to join our team in the role of Architecture Manager . · Busines ...
-
Arup Guangzhou, 中国Dedicated to sustainable development, Arup is a collective of designers, advisors and experts working globally. Founded to be humane and excellent, we collaborate with our clients and partners using imagination, technology, and rigour to shape a better world.' · To expand our bus ...
-
HSBC Guangzhou, 中国 À durée déterminée - temps pleinSome careers have more impact than others. · If you're looking for a career where you can make a real impression, join HSBC and discover how valued you'll be. · We are currently seeking an experienced professional to join our team in the role of Senior Software Engineer. · Bus ...
-
Control Engineer
2周前
Corning Guangzhou, 中国Major Responsibilities: · Assures safety of operators and equipment and Compliant with safety and environmental regulations. · Supervise Facility FMCS system which including PLC, HMI, iFix and database system management. · Responsible for Instrument selection and verification and ...
-
Avery Dennison Guangzhou, 中国 全职Job Description · ABOUT YOUR ROLE 职位描述 · The Senior Process engineer is in a position to lead the continual improvement of the production team, including process optimization, new equipment study and deployment, etc. The Senior process engineer is also responsible to collaborate ...
Data Engineer - Guangzhou, 中国 - Standard Chartered Bank
描述
Role Responsibilities
Job Role
We are seeking a highly motivated Data Engineer to join our dynamic Data Engineering team. As a Data Engineer, you will play a crucial role in managing and optimizing our data infrastructure, ensuring the reliability and availability of data for our analytics and business intelligence needs. You will work with cutting-edge technologies including Amazon S3, Amazon Athena, Apache Airflow, and Python to build and maintain data pipelines.
Key Responsibilities
Data Pipeline Development
Data Quality Assurance
Amazon Athena Expertise
Automation
Python Programming
Data Architecture
Documentation
Troubleshooting
Collaboration
Regulatory and Business Conduct
Our Ideal Candidate
Preferred Qualifications (not mandatory but a plus)
Role Specific Technical Competencies
Guangzhou的Data Engineer从业者
-
MARIKRISHNAN RAJAN
Marikrishnan Rajan (Mari), an Indian passport holder. Married since Jan-2020. · Bachelor of Enginee ...