Happy Deepawali and Happy Halloween
I am so happy to share this Diwali edition! It’s my favourite festival - not just a single day but a celebration that starts weeks in advance, with everyone meeting friends and family, exchanging sweets and gifts, and lighting up homes, apartments, and even markets in the most spectacular way. The vibe all around is so positive and cool!
Wishing you all a very Happy Deepawali! We celebrate by lighting up our homes with lamps and diyas to honor Lord Ram’s return to Ayodhya and the victory of good over evil. People also worship Goddess Lakshmi, the deity of wealth and prosperity, welcoming fortune and happiness into their homes.
Interestingly, Halloween falls on the same day this year! Though it’s not celebrated here at the same level as in the West, I’ve noticed that many kids dress up in spooky costumes and go house-to-house asking, ‘Trick or treat?’
Connected to that, this Halloween, I launched a special spooky Kubernetes series on Kubesimplify, presenting Kubernetes troubleshooting scenarios in a fun, eerie way. It was a challenge since it meant wearing costumes and staying in character for every video! A big shoutout to my wife, Saloni, for the concept, idea, and video editing. This series includes three videos with various scenarios → keeping learning at the center, but adding a twist of spooky fun!
Few updates from me
Been very active on Bluesky lately, due to the cloud native community coming by storm, you can connect with me here.
KubeCon NA is so close and you should not miss the session I am giving, If you are coming, do come to say HI!
This edition of the newsletter is brought to you by CastAI, and I would really like to thank them for sponsoring the time spent writing it. Crafting quality content takes time, and you should definitely check out CastAI if you're looking to reduce Kubernetes costs.
I also created another deep dive video on Kubernetes authentication and authorization to showcase how it works within Kubernetes, how to create users, generate KubeConfig files, and use `kubectl auth can-i` to check the required permissions and roles for each user.
What have I been reading?
The future of software is Nix - Determinate Systems envisions a future where Nix simplifies software development, offering a smooth, secure, and efficient experience by removing the complexities that hinder adoption. They are dedicated to creating tools, documentation, and workflows that make Nix accessible and reliable for developers and enterprises, ultimately enabling safer and more efficient software deployment for critical systems worldwide. I agree to this and that is why BuildSafe is built on top of Nix.
Running Phi 3 with vLLM and Ray Serve - This post delve into the differences between inference and serving and explore how to deploy the Phi-3 model using vLLM with Ray Serve on Kubernetes, a general-purpose scalable serving layer built on top of Ray.
State in Infrastructure as Code tools.- Why do Terraform, Pulumi, Helm, and other IaC / K8s tools store state? What challenges does it cause? What are the primary sources of those challenges?
Right-Sizing GPU and CPU Resources for Training and Inferencing Using Kubernetes - it highlights using Kubernetes for managing AI/ML workloads by dynamically optimizing GPU and CPU resources, particularly balancing intensive model training and frequent inferencing needs. Leveraging tools like Prometheus for monitoring and scripts for real-time resource adjustments, organizations can efficiently allocate resources, reduce waste, and enhance performance across cloud and on-premise environments.
Announcing Vitess 21 - Vitess 21 is live, focused on enhancing query compatibility, improving cluster management, and expanding VReplication capabilities
Awesome Resources/Repos
Fast Kubernetes - This repo covers Kubernetes with LABs: Kubectl, Pod, Deployment, Service, PV, PVC, Rollout, Multicontainer, Daemonset, Taint-Toleration, Job, Ingress, Kubeadm, Helm, etc.
NotebookLlama - Open source version of NotebookLM
Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)
Learn from X platfrom
https://x.com/danielepolencic/status/1848365836983279845
https://x.com/BenjDicken/status/1847310000735330344
https://x.com/llama_index/status/1850572786521256392
Do subscribe if you learned something new and also checkout CastAI and make their support worth it :)