Software Engineer (Pipeline Services) / Data Platform

求人概要 / Role and Responsibility
■ About the department
LINE Corp is a provider of its renowned messaging service and a wide range of other services, including finance, content and AI, to our hundreds of millions of global users. These services generate large amounts of data every day, resulting in LINE accumulating over 250 petabytes of data. As stated in "LINE STYLE", which defines the LINE practice and mindset, we take an “always data-driven” approach and leverage data as an asset across the company.
The Data Platform Department aims to democratize data and develop robust machine learning infrastructure. To this end, the department offers a platform that can leverage an enormous amount of data efficiently to drive service growth while also helping engineers, service planners, and marketers to capitalize on it.

■ About the job
As a Software Engineer for Pipeline Services, you will be developing a large scale data pipeline commonly used across the company that connects LINE's various services and Information Universe (IU), our proprietary data platform for data collection and analysis.

We build two types of data pipeline: one using Apache Flink for data streaming and the other using Django and Spark / Sqoop for snapshot batch data in database. The data pipelines are deployed on Kubernetes having as many as 1,000+ nodes and automated based on ArgoCD. You will play a key role in providing an architecture that allows flexible data design and advanced scale-out strategy in order to accommodate a variety of service and business needs, thereby supporting LINE services' sustainable growth in a data-driven manner.

・Dataflow capable of high availability and high throughput
・Incremental processing for large scale data
・Consistent CI / CD pipeline throughout from development to release
・Protocol design enabling flexible logging

This is a challenging position as you will be expected to understand and solve complex problems involving over petabyte-scale data. You will be provided with many opportunities and support to develop your skills and abilities to tackle organizational challenges through regular meetup events for data engineers and OJT.

■ Tools/development environments
・Streaming processing: Flink / Fluentd / Kafka
・Batch processing: Spark / Sqoop / Airflow
・Distributed environment: Kubernetes / YARN
・CI / CD: ArgoCD / DroneCI / Jenkins
・Operation and monitoring: Ansible / Grafana / Prometheus / Promgen
・Distributed storage: HDFS / Elasticsearch
・Query engine: Hive / Trino / Spark
・BI tools: Tableau / yanagishima / OASIS (internal BI tool)
・Development environment: IntelliJ / Github
・Language / Framework: Java / Kotlin / Scala / Python

■ Reference information
Flink@Data Platform - Ingestion Pipeline Redesign and Auto-scaling
応募資格 / Qualifications
■ Required experience and skills
・Experience developing/operating/troubleshooting a system in Linux environment
・Understanding of computer science fundamentals including data structure, algorithms, and computational analysis

■ Preferred experience and skills
・Experience developing streaming data pipelines using Flink / Spark
・Experience developing batch pipelines using Airflow, Luigi, Azkaban, Digdag, etc.
・Exposure to large scale distributed Big Data platforms such as Hadoop, AWS EMR, Cloud Dataproc, and machine learning pipelines
・Ability to build a system using container technologies such as Kubernetes
・Experience developing applications based on a message broker such as Kafka, AWS Kinesis, Cloud Pub / Sub
・Experience automating / optimizing operations using orchestration or monitoring tools
・Active contributions to open source projects

■ Ideal candidate
・Capable of proactively finding and solving problems
・Interested in distributed systems and data
・Have a willingness and desire to learn new technologies and be eager to take on new challenges
・Capable of capturing diverse user needs
・Able to involve and coordinate with other teams when necessary
・Able to tackle and solve difficult / complex issues
勤務地 / Location
四谷オフィス
〒160-0004 東京都新宿区四谷1-6-1四谷タワー23F
雇用形態 / Employment type
正社員
勤務時間 / Working hours
専門業務型裁量労働制(1日の労働時間に関わらず1日9.5時間労働したものとみなします。)、フレックスタイム制(コアタイム 11:00~16:00)、10:00~18:30(実働7時間30分)のいずれか適用 ※面接後に決定
待遇・福利厚生 / Benefits
■休日 / 休暇
・完全週休2日制(土・日)、祝日、年次有給休暇、年末年始休暇、慶弔休暇、リフレッシュ休暇制度(勤続5年で10日間)

■給与
・年俸制(保有されるスキル・経験・能力により優遇、相談の上で決定)
 - 年俸の12分の1を毎月支給。
 - 別途、インセンティブプラン有(※1)
・給与改定:年2回
・諸手当:交通費支給(会社規定による)、LINE Pay Card Benefit Plan(※2)

(※1)年俸額の他に、会社・本人の業績、評価に応じ、年2回インセンティブを支給することがあります。
(必ず支給されるものではありません。また、支給日に在籍している場合に限り支給対象となります。)
(※2)社員の「健康維持増進」「自己啓発」「次世代育成」を目的として給与とは別に支給する手当です。

■各種保険
・雇用保険、労災保険、健康保険、厚生年金保険

■福利厚生
・定期健康診断
・各種社内イベント他

■受動喫煙防止措置の実施
・屋内原則禁煙(屋内に喫煙室あり)
 - 四谷オフィス、南新宿オフィス、大崎オフィス

※当社はオフィス勤務と在宅勤務を組み合わせた、より効率的に高いパフォーマンスを発揮し続ける新しい働き方「LINE Hybrid Working Style」を採用しています。

その他詳細は面談の際にお伝えします。

■LINEに興味がある / マッチする職種があれば提案してほしい方はこちら
→キャリア登録について