Senior Software Engineer, Big Data
at Magnetic
This job is no longer active.
Magnetic is a technology company with a marketing platform for enterprises, brands and agencies. Our prospecting, remarketing, and merchandising solutions help marketers find, keep and bring back customers across channels and devices. These solutions are powered by our unique data including purchase intent and behavioral insights.
Our corporate headquarters, located in Silicon Alley, includes a fully stocked kitchen, our very own pool hall for an afternoon billiards match, and a beautiful office space designed to showcase our culture of transparency and collaboration. And if youre an urban outdoors enthusiast, Madison Square Park and all it has to offer is just one block away!
About the role
Magnetic is a respected marketing and advertising technology company with established lines of business. The company is aggressively investing in expanding its high-quality data science and decision automation capabilities.
Our current AdTech platform serves up hundreds of thousands of real-time ads each second. Our complementary MarTech CRM platform provides billions of product recommendations to hundreds of millions of web shoppers across many of the webs most popular retail sites. We are bringing these two capabilities together based on an improved unified view of the customer and radically upgrading our decision automation. This upgrade activity represents a substantial re-architecting of our existing platform and is expected to accommodate well in excess of 1 million queries per second. From a technology perspective this is a challenging environment requiring millisecond latencies from a distributed service oriented architecture in AWS. The mission is simple, to extract as much predictive power as possible from the incoming real-time data streams, and apply that power via optimizations, in a way that delivers greatest commercial value to our clients and the company. Our solutions are only as good as our answers and we hold ourselves to account in everything we do by using extensive performance benchmarking.
We use many of the high-throughput and big data technologies that one might expect including: Spark, Kafka, Hadoop, HBase, Pig, Impala, Parquet, Memcached, VW, XGBoost, LibFM, and others. Our preferred coding languages are Python, Java and Scala. As a Senior Engineer, Big Data and Data Science, you will help us understand the intents of consumers, and our ability to influence outcomes, as expressed through their digital interactions. You will work with product management and data science teams to build resilient smart products that make informed decisions in real time based on the data.
Characteristics of the likely successful candidate:
- An outstanding engineer from a good university
- 5+ years of experience in software development, a substantial part of which was gained in a high-throughput, decision-automation related environment
- Broad experience in manipulating big data using technologies like Spark, Kafka, Hadoop and Impala
- Experience with AWS
- Experience deploying solutions for data-science-driven decisioning at scale using technologies such as VW, XGBoost and LibFM
- Produce high-quality code in Python, Java, Scala or all
- Passionate about testing, and with extensive experience in Agile teams using SCRUM you consider build automation to be the norm
- An independent thinker who considers the operating context of what he/she is developing
- Believes that the best data pipelines run unattended for weeks and months on end.
- Follows data development best practices, and enjoy helping others learn to do the same.
- Well-versed in (or contributes to) data-centric open source projects.
- Reads Hacker News, blogs, or stays on top of emerging tools in some other way
- Proven ability in a high-performance, collaborative environment
Skills
Java
Python
AWS
Agile
Hadoop
Scrum
Scala
Kafka
Spark
Data Engineering
Data Pipeline
This job is no longer active.