Apache Arrow has become foundational infrastructure for the modern data stack. With massive amounts of data moving through Arrow in an incredibly reliable way, it became easy to assume the infrastructure would just be there to support the workloads without intervention. In 2025, that assumption was tested.

A leading marine science center needed to connect ocean biogeochemical models to fine-grained microbial metabolic models, but available data was fragmented across six incompatible sources.

Communicating the positive contribution of your OSPO operates on multiple levels simultaneously. Whether you're talking to executives, engineering management, or engineers, each audience needs to hear about the OSPO’s impact in terms that matter to them. When it comes to securing the OSPO’s actual budget, though, what matters most is communicating to your executive team in clear and validated dollar values.

A quantitative trading customer lacked the in-house HPC specialists needed to build batch tooling around their systems. They suffered cost overruns on their cloud HPC workloads due to an inability to estimate compute costs before running a job.

Apache Arrow lost its main corporate sponsor in 2024, jeopardizing the employment of the Arrow community's core contributors. This was disastrous for Arrow and Parquet, technologies that underpin a huge proportion of data-related projects today, and the companies that depend on them.

A fleet safety software company needed external validation of their machine learning and data engineering architecture. Their system analyzes data from diverse, incompatible sources (cameras, accelerometers, accident reports, violation records, and more) to predict driver risk.

A major automaker needed comprehensive route analysis data to support an electric truck grant proposal, requiring precise measurement of diesel truck miles driven through California's Disadvantaged Air Quality Communities (DACs) across multiple air districts.

A customer needed to explore a vast design space for high-performance structures with fluid dynamics constraints, but traditional CFD modeling was too computationally expensive and time-consuming to iterate through all possible permutations of design.

An industrial seals distributor struggled with unpredictable customer demand patterns, manual forecasting processes, and costly inventory misalignments due to significant gaps between customer forecasts and actual orders.

A semiconductor equipment parts manufacturer struggled to expand their total addressable market, citing disconnected sales processes and institutional knowledge trapped in experts' heads rather than accessible systems.

A small discretionary trading hedge fund was operating with fragmented data systems, making it impossible to generate a unified view of portfolio risk.

A manufacturing company relied entirely on on-premise servers, networks, and desktop computers, creating operational vulnerabilities and high maintenance costs.

BMW and PG&E needed advanced data infrastructure and simulation capabilities to support EV charging infrastructure placement, as well as a consumer-facing app helping drivers optimize their charge times and reduce greenhouse gas emissions.

A customer needed to scale their HPC installation after completing their initial research phase. They had a team of top-tier scientists, but lacked the HPC infrastructure specialists needed to configure their cutting-edge hardware.

A rubber seal manufacturer needed to modernize their sales operations. We helped them transform their systems from a series of disconnected spreadsheets and over-reliance on institutional knowledge into a structured, data-driven system that allowed them to increase their addressable market size.

A nuclear fusion startup came to us with three business problems: Their computational fluid dynamics (CFD) simulations were taking too long to run, they were too expensive, and these problems were capturing too much of the CEO's attention. Keep reading to learn how we helped them solve all three problems at once.

Last week, the CA Governor's Office of Business and Economic Development (GO-Biz) released the $27M earmarked for data interoperability and technology upgrades for the five containerized ports in California. This grant program, which we designed and implemented along with our partners at GO-Biz and Momentum, is designed to ensure that our containerized ports, which process about 40% of all containerized imports in the U.S., have the technology needed to remain competitive while also driving statewide economic, supply chain, and environmental benefits.

No matter the methodology used to calculate its value, it’s certain that Open Source Software is an incredibly valuable commodity. The question is how to take that fact and apply it practically within an organization. In this article, I'll show you some methods I have developed to effectively measure the value created by the large OSS team that I lead for G-Research.

ISC celebrated the milestone of awarding all the funds to the five participating ports by taking a trip to the Port of Oakland, to meet some of the project's stakeholders in person and to witness the immense scale of this world-class facility.

Every company needs data science and AI — or so they think — while at the same time most data science projects fail to deliver any lasting value. While there is no exact measurement on the percentage of data science projects that fail, there is consensus that the failure rate is quite high. Read on to see some of the most common causes of data science project failure.

It’s impossible to watch the news or scroll social media lately without hearing about AI, often citing its dangers. Like any powerful tool, it has the potential to do harm. You could end up harming people (and your own business) if you don’t design your AI solution with this in mind. Here I’ll give you a high-level overview of three essential focus areas to help you utilize the power of AI in a safe way.

One reason ISC was excited to lead the scoring and evaluation portions of the CA Port Data Interoperability Grant Program was that it represented an opportunity to identify and implement improvements to the process. We strongly feel that the addition of a feedback round to the grant application process allowed the proposals to be stronger, more competitive, and more aligned with the goals and budget of the State of CA.

Prior to the last Supercomputing23 conference in Denver, there was a pre-conference workshop called NRG@SC23. The purpose of this eye-opening workshop was to bring together a diverse group of researchers from underserved regions to discuss the challenges of running large-scale HPC clusters with limited resources.

While learning to write code and looking at data seems the natural starting point of any analytics job, doing so leads to a high likelihood of failure.

We passed a big milestone last week with the CA Port Data Interoperability Grant Program. After providing draft feedback on proposals (the first time in the nation that such a grant program has included a feedback stage), the proposal submission window closed on Feb 8th. We will officially announce the awards on Feb 26th. The review process was designed to provide actionable feedback on high-level green/red flags, according to the scoring criteria developed in the initial phase of the project, in order to make the proposals stronger and more competitive.

What do Pythons and Pandas have to do with data? The data analysts and scientists at Insight Softmax use Python and Pandas every day to solve complex business problems.

On January 4th, the CA Port Data Interoperability Grant Program team held a roundtable discussion with all of the participating ports, led by the California Association of Port Authorities. Together, we shared proposal status updates, collaborated on non-containerized standards development, and worked towards supporting future funding opportunities.

In this 5-part series, we will follow our COO, Marcus Rabe, as he teaches a 6-week class, Computational Analytics with Python, at Hult International Business School in Boston, MA.

For Filip Novović, cloud infrastructure engineer at Insight Softmax, delving into complex topics and studying academic theories with his coworkers is something he loves about working at the data science company. But what’s even better is taking those ideas and using them to solve real-world business problems for clients.

The official proposal framework for the California Port Data Interoperability Grant Program has been released. As of Dec. 12, the five containerized ports in California can submit grant proposals under the program, which is charged with awarding $27,000,000 in total funding for data system modernization and interoperability improvements.

ISC is excited to announce that we’re leading a pioneering project, the first of its kind in the US: the California Port Data Interoperability Grant Program. The overarching goal is to improve data system interoperability between the ports and their stakeholders through a series of grants totaling $27,000,000.

How businesses create value from their data: Surround it with context, transforming it into valuable information that can be used to make smart business decisions.

What do space physics and data science have in common? As it turns out, a lot. George Geddes, a data scientist at Insight Softmax, uses the knowledge he gained from his space-based PhD research to solve earthbound business problems every day.

If you’ve ever uttered “Facebook made me buy it,” you know the power of a good recommendation engine. When the algorithm is on point, the tool suggests products and services to you that you didn’t even know you needed.

Logistics is one of the planet’s most essential – and most complicated – industries. There’s lots of room for error when you’re dealing with a (literal) million moving pieces scattered around the planet, but data science gives logistics companies the opportunity to connect those dots.

We’re making great strides in quantum computing, but it’s important to remember the end goal: turning our theoretical research into real-world solutions for business applications.