1/5 Just made my first contribution to the #Datafusion #Comet - a native physical execution engine for #Apache #Spark! 🚀
While Spark with it's row oriented model and code generation approach is quite good on average, there is almost always a faster specific solution.
While Spark with it's row oriented model and code generation approach is quite good on average, there is almost always a faster specific solution.
Comments
#Rust #Datafusion
It is not supported in Datafusion, so I had to write it with low-level Arrow. It was a challenge to handle all the corner cases (negative indices, boundaries, etc.). Thanks to rust-gdb, which was surprisingly friendly for a CLI tool!
#Rust #ApacheArrow
https://semyonsinchenko.github.io/ssinchenko/post/comet-first-contribution/