Ziaul Kamal on LinkedIn: 4 keys to writing modern Python https://lnkd.in/gfTxqiYr Although Python… (2024)

Neglect Kubernetes Resource Management at Your Perilhttps://lnkd.in/gfTxqiYrWhen I opened our first Kubernetes cluster to hundreds of users in 2016 as the platform engineering manager of a large bank, I borrowed the phrase “benign neglect.”That supposedly benign neglect was about managing Kubernetes resource primitives. While I understood that migrating to Kubernetes allowed us to constrain the minimum and maximum resources used by each container, something we could not do on our legacy grid computing system, it presented many new challenges. First and foremost, I needed to determine appropriate values to set for those constraints. Subsequently, I needed to educate users on how and why to use them.I decided to set some loose defaults to start, and I hoped that the “Kubernetes invisible hand” would magically take care of the compute resources until someone came along with a better solution that would scale efficiently. It took only a few days for my plan to backfire, with users complaining that they could not schedule pods due to a lack of resources. Meanwhile, the internal virtualization team called me, confused about how the entire cluster had only 15% CPU utilization on the virtualization layer, yet hundreds of users were unable to access the available resources.This was how I learned that my neglect was far from benign. My Kubernetes journey was off to a rough start.Why You Need to Manage Kubernetes ResourcesAvoiding the widespread issues in my Kubernetes rollout required a nuanced approach to managing Kubernetes resources. Specifically, I needed a different strategy for managing the two key settings that Kubernetes provides to constrain resources.In Kubernetes configuration, requests are used to set the minimum resources a container is guaranteed to access, while limits constrain the maximum resources a container can consume on a node. Together, they allow users to isolate CPU and memory. While these are conceptually straightforward, finding the optimal values can be extremely challenging — especially at scale.My experience in 2016 revealed a few major problem areas that result from not managing resource requests and limits effectively. The first is poor reliability and unpredictable performance. When workloads request too few resources, they’re underprovisioned, leading to resource contention on nodes (which yields CPU throttling, out-of-memory kills and pod eviction). The second is high cloud costs. When workloads request more resources than they need, they’re overprovisioned, leading to low node utilization, which translates to cloud bills that are two to three times higher than necessary.As I progressed down what I now see as the typical resource management journey, I discovered a third central problem area: overwhelming engineering teams with toil. At scale, manually tuning resources quickly overwhelms both developers and platform teams.Learning about this the hard way was a messy experience. Let’s take a ...

Java’s Project Leyden arrives in early-access buildhttps://lnkd.in/gfTxqiYrThe OpenJDK’s Project Leyden, an effort to improve the startup time, time to peak performance, and footprint of Java programs, has published its first early-access build.Accessible from java.jdk.net, the initial Project Leyden release focuses on improving application startup time, according to a June 24 bulletin on the Oracle Java team’s Inside Java website. Other Leyden optimizations will be explored in future releases. The easiest way to start with Leyden features is by using the java program in the Leyden Early Access Release with the -XX:CacheDataStore flag. Project Leyden is sponsored by Java’s HotSpot and Core Libraries groups.To read this article in full, please click here

Human Insight + LLM Grunt Work = Creative Publishing Solutionhttps://lnkd.in/gfTxqiYrIf you publish product documentation to a website, this might be a familiar pattern. You write an initial draft in Google Docs so your team can review and propose changes. It’s a rich authoring environment in which you can insert images by pasting bits copied from screens, and easily create and rearrange tables. When the team reaches a consensus, you have a nice representation of the web page you’d like to appear on your site.But how do you make that happen? In this post, we’ll look at a surprisingly simple method that’s proven effective. In our case it’s tuned for a workflow that pushes GitHub repositories into Next.js-based sites hosted by Vercel, so our Markdown text and associated images are copied into those repos. But the method will work for any Markdown-oriented publishing system.Why not use Markdown syntax in Google Docs and pass it through to publishable Markdown?The naive solution is, of course, to just export the Google Doc as HTML. That gets you most of the way there, but the last mile is a slippery slope. Google Docs won’t let you create custom styles to align elements with their counterparts in your published web page. And the images are sourced from googleusercontent which is convenient — they seem to just magically work — but you probably want those images stored as named files in your publishing system.So what I’ve seen happen, in several different environments, is manual transfer and reformatting that becomes a tax on the collaborative benefit of Google Docs.I’d taken a few runs at this problem in the pre-LLM era, so for starters I revisited what my new team of assistants might bring to the party. Feeding an HTML export from Google Docs into Python’s Markdownify got us tantalizingly close, but that last mile really is a slippery slide into the weeds of document conversion.Then it occurred to me: people are already writing triple backticks for code blocks, single backticks for inline code, ** and * for bold and italic, and Markdown-style links and lists. Why not use that Markdown syntax in Google Docs and and pass it through to publishable Markdown? That turned out to be a really fruitful idea that was easy to implement.Progressive Pattern SimplificationHere’s the syntax I’m using to identify images in Google Docs.[image: image_1 70%]Here’s how that looks in the exported HTML.[image: image_1 70%]<br></span><span style=”overflow: hidden; display: inline-block; margin: 0.00px 0.00px; border: 0.00px solid #000000; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px); width: 620.50px; height: 648.64px;”><img alt=”” src=”https://ift.tt/thkG3OJ″ style=”width: 620.50px; height: 648.64px; margin-left: 0.00px; margin-top: 0.00px; transform: rotate(0.00rad) translateZ(0px); -webkit-transform: rotate(0.00rad) translateZ(0px);” title=””>Here’s the...

TypeScript 5.5: Faster, Smarter and More Powerfulhttps://lnkd.in/gfTxqiYrMicrosoft recently released TypeScript 5.5 offering a range of features and optimizations to enhance the company’s JavaScript superset.This update includes inferred type conditions, improved expression validation and separate declarations, as well as notable performance boosts and enhancements to editor reliability.Better Developer ExperienceThe release focuses on improving developers’ experiences. TypeScript 5.5 aims to provide quicker build processes and stronger tooling assistance.“Writing types in our code allows us to explain intent and have other tools check our code to catch mistakes like typos, issues with null and undefined, and more,” wrote Daniel Rosenwasser, principal product manager for TypeScript at Microsoft, in a blog post. “Types also power TypeScript’s editor tooling like the auto-completion, code navigation, and refactorings that you might see in editors like Visual Studio and VS Code. In fact, if you write JavaScript in either of those editors, that experience is powered by TypeScript!”Microsoft has made several changes to the language since the beta and release candidate versions of TypeScript 5.5.For instance, “since the beta, we added support for ECMAScript’s newSetmethods. Additionally, we’ve adjusted the behavior ofTypeScript’s new regular expression checkingto be slightly more lenient, while still erroring on questionable escapes that are only allowed per ECMAScript’s Annex B,” the post saidMicrosoft also added and documented moreperformance optimizations: notably, skipped checking in transpileModule and optimizations in the way TypeScript filters contextual types. These optimizations can lead to faster build and iteration time, the company said.Summary of Key New FeaturesSummarized highlights of key new features and improvements in TypeScript 5.5 include:Inferred type predicates: Improves type inference in certain scenarios, especially with arrays and filtering.Control flow narrowing for constant indexed accesses: Enhances type narrowing for object property accesses.The JSDoc @import tag: New tag for importing types in JavaScript files without runtime impact.Regular expression syntax checking: Basic syntax checking for regular expressions to catch common mistakes.Support for new ECMAScript Set methods: Adds declarations for proposed new Set methods.Isolated declarations: New compiler option to help with faster declaration file generation.The ${configDir} template variable: Helps with writing more portable configuration files.Consulting package.json dependencies: Improves declaration file generation by considering package dependencies.Editor and watch-mode reliability improvements: Various fixes to improve editor experience and watch mode.Performance and size optimizations: Multiple improvements to compiler speed and package size.Easier API consumption from ECMAScript Modules: Better suppor...

How to Achieve Ultimate Freedom with Your Load Balancerhttps://lnkd.in/gfTxqiYrThe popularity of open source software continues to soar alongside growth in cloud usage, including multicloud and hybrid cloud infrastructure. Pluralsight’s State of Cloud 2023 report revealed that 65% of organizations actively use multicloud environments.Open source is prized, in part, because it enables freedom of choice in an organization’s tech stack, especially in complex deployment environments. The need for choice extends to load balancers, which are indispensable infrastructure components, powering rapid, reliable and secure application delivery.Load balancing software handles a wide range of functions and interacts with many other components. Flexibility at the load-balancing level helps maximize freedom of choice across your entire application infrastructure.Load balancers are vital to application delivery, which is supported by scalable, distributed systems. As client requests arrive — for instance, to display a product page to a user — the load balancer distributes the requests to backend servers according to specified load-balancing algorithms. This prevents servers from becoming overloaded, boosting overall capacity and helping applications scale in a cost-efficient manner.That was all traditional load balancers did, but modern load balancers have evolved into robust application delivery solutions, with additional features such as:Advanced request routingFailover protection and health checking, which help to ensure high availabilitySecurity capabilities, including next-generation web application firewall (WAF) capability and bot managementCentralized management, monitoring and automation, resulting in greater operational efficiencyOrganizations need a feature-rich load balancer that integrates seamlessly everywhere — elevating load balancers beyond simple tools to flexible, all-in-one workhorses for modern application environments.Here are key characteristics that distinguish a modern load balancer capable of maximizing freedom in your tech stack:Open Source FoundationsProduct development rooted in community involvement and open source contributions is powerful. While many solutions are black boxes, providing few hints as to how they’re built or function, a transparent and customizable solution built upon feedback will serve users best. This approach is more responsive to user needs and bolstered by peer review. Look for healthy and robust open source projects with a demonstrated history of community code contributions.Widespread Popularity Over TimeWidespread popularity indicates a vendor is supportive of customer needs over the long term, which enables customers to build and innovate from a position of trust. We see this happen in a couple of ways.First, a widely used and enduring project matures and improves quickly and consistently, having been exposed to a plethora of challenges and changing technologies....

DataStax updates tools for building gen AI applicationshttps://lnkd.in/gfTxqiYrDataStax is updating its tools for building generative AI-based applications in an effort to ease and accelerate application development for enterprises, databases, and service providers.One of these tools is Langflow, which DataStax acquired in April. Itis an open source, web-basedno-codegraphical user interface (GUI) that allows developers to visually prototypeLangChainflows and iterate them to develop applications faster.To read this article in full, please click here

Maximizing Terraform Modules for Platform Engineeringhttps://lnkd.in/gfTxqiYrTerraform modules encapsulate complex configurations into reusable, manageable components. Many teams use these modules to streamline infrastructure management, significantly boosting productivity.By using prebuilt modules as templates, teams avoid the repetitive task of writing infrastructure code from scratch for each new project, allowing them to focus more on developing the application itself. This practice enhances consistency and reduces the likelihood of errors by reusing proven configurations across different environments.However, many teams use Terraform modules in a way that misses their full potential and leads to maintenance challenges and inconsistencies. Let’s look at two common practices, their challenges and the changes teams can make to harness the full power of Terraform modules.Challenges With Current Terraform Module UsageCommon Practice 1: Rigid Starter TemplatesUsing prebuilt templates for new projects is appealing for quick setups. Tools like Backstage offer catalogs of starter projects, providing predefined configurations that are often composed of Terraform modules. However, these templates are usually rigid, covering general use cases rather than the specific needs of evolving projects. As project requirements change, maintaining and updating these modules becomes increasingly complex. Developers must manually modify configurations, which leads to inconsistencies and versioning issues, and results in outdated or insecure setups.Common Practice 2: Tight Coupling of Infrastructure and Application CodeMany teams have a single comprehensive Terraform project that defines the entire infrastructure setup of their application. This creates a tight coupling that leads to a complex deployment pipeline, where any infrastructure change requires a full redeployment. Such tight coupling creates dependencies that can slow down the release pipeline and make infrastructure changes difficult to manage efficiently.The reliance on rigid templates and the tight coupling between application code and infrastructure configurations keep teams from fully leveraging Terraform modules in modern infrastructure management.To overcome these challenges, teams should shift their perspective to build a true platform. Instead of using static code snippets and comprehensive Infrastructure as Code to manage manually, teams should focus on dynamic, reusable components. This is made feasible by introducing a framework capable of orchestrating the linkage of Terraform modules to application requirements in an automated way.Enhancing Modularization With Infrastructure from CodeInfrastructure from Code (IfC) is a new paradigm that helps teams achieve modularization; it complements Terraform by offering an intelligent framework that automates infrastructure provisioning based on application requirements. By dynamically identifying and applying modules...

A balanced approach to AI platform selectionhttps://lnkd.in/gfTxqiYrI’m not sure why our industry keeps falling into the trap that when a new concept emerges, there are near-immediate announcements that it runs best on one platform. Enterprises shouldn’t even think about other options.This VentureBeat article is an example, although it is more balanced than most. While many pundits present cloud computing as the only rational choice for AI, many hardware vendors declare that traditional hardware is the best option. Who’s right?The nuances of platform selectionThe questions I get at AI speaking events used to be some version of “what’s the best cloud?” Now it’s “where should I run AI?” Neither question has a black-and-white answer. A lot of planning must go into the selection process to define the best clouds and best AI platforms to solve specific problems.To read this article in full, please click here

AI is in the tire-kicking phasehttps://lnkd.in/gfTxqiYrJust like big data back in 2013, we’re in the “everyone’s doing it, no one knows why” phase of generative AI (genAI). A recent McKinsey survey found that 65% of enterprises are “regularly using genAI.” Promising! In Elastic’s recent earnings call,the company noted that over 1,000 customers are paying to build genAI applications. Wow! Each of the big cloud companies, as well as Oracle, has talked up how genAI is driving cloud spend. Amazing!To read this article in full, please click here

What senior developers knowhttps://lnkd.in/gfTxqiYrYou will find a lot of articles on the web about what it takes to become a senior developer. Many of these articles focus on soft skills. They often advise staying current with technology, learning to communicate well, mentoring junior developers, and other things along those lines. These recommendations are all fine and good, but, well, let’s be honest. They are also pretty much pabulum.I’d like to add a little depth to the discussion. The essential difference between senior developers and junior developers is time in the software development trenches, and what they’ve learned from it. With that in mind, I’ve come up with a far-from-exhaustive list of four things the senior developers know, usually through hard-earned experience.To read this article in full, please click here