Systems PhD - Software Engineer

<p>P-1299</p> <p>Databricks is radically simplifying the entire data lifecycle, from ingestion to generative AI and everything in-between. We’re doing it cross-cloud with a unified platform, currently serving over 10k customers, processing exabytes of data/day on 15+ million VMs, and growing&nbsp;<em>exponentially</em>.&nbsp;</p> <p>To make it happen we’re building multi-cloud systems at every corner of the data ecosystem, from query engines, vector databases, training pipelines, and storage systems, down to the infrastructure that allows them to scale like auto-sharders, caches, and load balancers, just to name a few. We also build and support the tooling, languages, and stacks that bring it together. Basically, we do it all.&nbsp;</p> <p>The space we work in and the problems we solve are massive, complex, and very deep (our published work on<a href="https://cs.stanford.edu/~matei/papers/2021/cidr_lakehouse.pdf"> Lakehouse</a>,<a href="https://www.vldb.org/pvldb/vol13/p3411-armbrust.pdf"> Delta lake</a>, and<a href="https://cs.stanford.edu/~matei/papers/2022/sigmod_photon.pdf"> Photon</a> are a testament to that). We’re looking for practitioners who are eager to work with the best in industry to push the boundaries of what’s possible for our customers. If you’re truth seeking, data driven, and love to operate from first principles (head fake: <em>our core values</em>), then Databricks is the place for you.</p> <p>&nbsp;</p> <p>As a part of the Database Engine team, there are opportunities to design and implement in many areas that leapfrog existing state-of-the-art systems:</p> <ul> <li>Query compilation &amp; optimization</li> <li>Distributed query execution and scheduling</li> <li>Vectorized engine execution</li> <li>Data security</li> <li>Resource Management</li> <li>Transaction coordination</li> <li>Efficient storage structures (encoding, indexes)</li> <li>Automatic physical data optimization</li> </ul> <p>&nbsp;</p> <p><strong>What we look for:</strong></p> <u