Greybeam

What's the one thing engineers LOVE about dbt Labs? This week Arsham and I ran around Coalesce armed with our magic microphone to ask a few burning questions. We cornered founders, engineers, and everyone in between to get their unfiltered takes. 🎥 Here’s part 1 of our mini-series. What's YOUR favorite thing about dbt? Drop it below 👇 Massive thanks to the brave souls who let us ambush them: Oliver Laslett, Rick Radewagen, Christophe Blefari, William Tsu, Jon Johnson, Benjamin Segal, Chloé L., Ben Lerner, Melanie, Jericho Villareal, David S., and Jay 🙏

Arsham and I are headed to Vegas for Coalesce this week! Would love to meet up with folks for a coffee and a chat if you'll be there.

pro tip: don't use a 4XL warehouse unless you want to be homeless I guess it's hard to trick Arsham if he doesn't check his phone...

This is why you should never SELECT *. Snowflake and DuckDB are columnar databases. They store each column separately. When you only need 3 columns from a 80-column table, they should only read those 3 columns from disk, this is called column pruning. Most query optimizers, like the ones in Snowflake and DuckDB, will automatically detect columns that are referenced but not used in a query and prune them. For example, in the query below, the optimizer should know to only retrieve name from the customers table despite calling SELECT * in the subquery. SELECT name FROM (SELECT * FROM customers) Most of the time, it should just work. But certain things can break it. I ran into this while testing DuckDB 1.4.0 (upgrading from 1.3.2) and noticed queries that once took 10s were now taking over 100s. The updated query optimizer was not pruning 60 unused columns from a 2 billion record table all because there was a SELECT * in an early CTE. Snowflake is guilty of the same, but the difference in execution time is not as stark. Try it yourself!

The cleaner your SQL looks, the worse it might perform. Take this perfectly readable join: FROM orders JOIN customers ON orders.customer_id = customers.id OR orders.legacy_customer_id = customers.id Looks elegant. Runs terribly. Why? Snowflake can't use hash joins with OR conditions! There is no longer one clear key to partition on and the query optimizer has to account for matches on either column. The ugly fix that actually works: SELECT * FROM orders JOIN customers ON orders.customer_id = customers.id UNION SELECT * FROM orders JOIN customers ON orders.legacy_customer_id = customers.id Less readable but nearly 40x faster. This OR join pattern shows up everywhere and it's just one of many innocent looking query patterns that destroy performance. I just wrote a blog that deep dives into 7 ways to make your queries run faster. The others include: - Join optimization - Query pruning - Proper use of warehouse cache - Choosing the right query engine If your queries are slow and expensive, these patterns might be why. 👇 https://lnkd.in/g_v7PjPM edit: the SF10 on the left is a screenshot error. If we run the same query on SF10 it would've taken 2h 57m!

I’m happy to share that I’m starting a new position as Founding Engineer at Greybeam!

To those attending the Snowflake Summit, be wary of mad men jumping over the stalls... They're trying to save you money. 💰 Using a big flush 🚽 for your small twinkles ✨ is costing you time and money. If toilets can have a small flush, why can't Snowflake? Most queries don't need massive clusters--but you're paying for them anyway. Greybeam gives you the missing small flush option. Light queries go to DuckDB, heavy analytics stay on Snowflake where they belong. Stop flushing money down the drain. And start saving up to 85% on Snowflake costs because you're finally using the right tool for the right job.

Have you always aspired to be like Smitty Werbenjagermanjensen 🧽 ? If the answer is yes, join us at Greybeam and be employee #1. 📍 𝗟𝗼𝗰𝗮𝘁𝗶𝗼𝗻: In-person in San Francisco 💵 𝗖𝗼𝗺𝗽𝗲𝗻𝘀𝗮𝘁𝗶𝗼𝗻: $160K - $200K + equity 📋 𝗪𝗵𝗮𝘁 𝘆𝗼𝘂'𝗹𝗹 𝘄𝗼𝗿𝗸 𝗼𝗻: backend heavy and you'll do a little bit of everything More details here 👉 https://lnkd.in/gnG9fAKK