LLM Agents Improve Supply Chain Planning by 22% in Accuracy

Research Background

In the rapidly evolving landscape of supply chain management, the need for efficient, accurate, and responsive planning is more critical than ever. The complexity of modern large-scale supply chains, driven by the explosive growth of e-commerce, demands sophisticated tools to handle the multifaceted challenges of demand forecasting, inventory management, and order fulfillment. Traditional approaches, while effective in static environments, often struggle to adapt to the dynamic and uncertain nature of today’s retail operations.

The problem at hand is the need for a robust, adaptive, and intelligent planning system that can handle the high-dimensional, real-time data and complex decision-making processes inherent in modern supply chains. This is particularly important for large e-commerce platforms like JD.com, which manage over 10 million SKUs across thousands of warehouses and distribution centers, serving 600 million annual active users. The sheer scale and complexity of such operations require a planning system that can not only process vast amounts of data but also make informed, timely decisions.

Why this matters is evident in the operational challenges faced by traditional planning systems. These systems often rely on rule-based approaches or optimization models, which, while effective in stable conditions, are ill-suited to the volatile and unpredictable nature of modern supply chains. Misaligned plans can lead to stockouts, overstocking, and service degradation, all of which have significant financial and reputational costs. The integration of AI, particularly Large Language Models (LLMs), offers a promising solution by enabling real-time reasoning, adaptive optimization, and automated decision-making.

The industry context is one of increasing sophistication in supply chain systems, driven by advancements in AI and machine learning. LLMs, with their ability to understand natural language, reason over structured and unstructured data, and orchestrate multi-step workflows, provide a new paradigm for supply chain planning. However, prior approaches have often been limited by their inability to handle the full spectrum of tasks required in a real-world supply chain, from data acquisition and plan formulation to execution and continuous refinement. The proposed LLM-based framework aims to address these shortcomings by providing a comprehensive, modular, and scalable solution.

Traditional planning systems, such as those based on Mixed Integer Programming (MIP) or heuristics, are effective in well-structured, static environments. However, they face several limitations in modern e-commerce scenarios. For instance, they struggle with data heterogeneity and incompleteness, where input data originates from various sources, including ERP, WMS, TMS, and third-party logistics systems, and often suffers from delays, noise, or missing values. Additionally, high uncertainty in demand patterns, driven by factors such as market trends, seasonality, and external disruptions, makes it difficult to forecast accurately and plan efficiently. Execution inefficiency and adaptation latency are other significant issues, as adjustments to accommodate new constraints or changing business priorities often require restarting the planning process from scratch, resulting in significant delays and inefficiencies.

Key Findings

Reduction in Data Processing Time

One of the key findings of the study is the significant reduction in weekly data processing and analysis time, which was reduced by approximately 40% per cycle. This improvement in human efficiency allows planners to allocate more time to other critical tasks, thereby enhancing overall operational productivity.

The method principle behind this finding is the modular and automated nature of the LLM-based planning assistant. By decomposing complex supply chain tasks into smaller, manageable sub-tasks, the system can efficiently process and analyze large datasets. The key design logic involves the use of an LLM to classify intents, orchestrate tasks, and execute them in a coordinated manner. This approach leverages the strengths of LLMs in understanding domain knowledge, comprehending operator needs, and generating evidence-based planning reports.

The experimental setup involved deploying the LLM-agent framework in JD.com’s real-world scenario, where it was tested on actual operational data. The results showed a 40% reduction in data processing time, as measured by the time taken to complete weekly data analysis cycles. This improvement is attributed to the system’s ability to automate repetitive and time-consuming tasks, freeing up human planners to focus on higher-value activities. Specifically, the LLM agent was able to handle data cleaning, preprocessing, and initial analysis, reducing the time spent by human planners on these tasks from an average of 20 hours per week to 12 hours per week. This 40% reduction in data processing time translates to a significant increase in operational efficiency, allowing planners to focus on strategic decision-making and other critical tasks.

Improvement in Plan Accuracy

Another significant finding is the notable improvement in the accuracy of generated plans. The proportion of plans with an accuracy deviation below 5% increased by 22%, indicating that the planning assistant can produce more reliable and precise recommendations.

The method principle here is the iterative plan correction mechanism, which continuously refines the generated plans based on intermediate results and feedback. The key design logic involves the use of an LLM to decompose complex planning tasks into smaller, modular sub-tasks, each of which can be executed and refined independently. This iterative process allows the system to learn from its mistakes and improve the accuracy of its recommendations over time.

The experimental setup included a series of tests on real operational data from JD.com, where the LLM-agent framework was used to generate and refine sales and inventory plans. The results showed a 22% increase in the proportion of plans with an accuracy deviation below 5%. This improvement is attributed to the system’s ability to dynamically adjust plans based on real-time data and feedback, ensuring that they remain aligned with actual business conditions. For example, the LLM agent was able to incorporate real-time demand signals and adjust inventory levels accordingly, leading to more accurate and reliable plans. The accuracy of the plans was measured by comparing the predicted demand and inventory levels with the actual demand and inventory levels, and the results showed a 22% increase in the number of plans with an accuracy deviation below 5%.

Increase in Stock Fulfillment Rates

The third key finding is the increase in stock fulfillment rates, which improved by approximately 2-3% relative to the previous level. This increase reflects enhanced efficiency and precision in inventory management, resulting in better alignment between supply and demand.

The method principle behind this finding is the system’s ability to optimize inventory allocation and replenishment planning. The key design logic involves the use of an LLM to analyze historical sales data, forecast future demand, and generate optimal inventory plans. By continuously monitoring and adjusting inventory levels, the system ensures that the right products are available in the right quantities at the right time.

The experimental setup involved deploying the LLM-agent framework in JD.com’s real-world scenario, where it was tested on actual operational data. The results showed a 2-3% increase in stock fulfillment rates, as measured by the percentage of orders fulfilled without stockouts. This improvement is attributed to the system’s ability to dynamically adjust inventory levels based on real-time demand forecasts and operational constraints, ensuring that inventory remains aligned with actual customer demand. For example, the LLM agent was able to predict spikes in demand and adjust inventory levels accordingly, leading to a 2-3% increase in the stock fulfillment rate. This improvement in stock fulfillment rates translates to better customer satisfaction and reduced operational costs.

Limitations

Data Heterogeneity and Incompleteness

One of the main limitations of the LLM-based planning assistant is its reliance on high-quality, complete, and consistent data. In real-world scenarios, input data often originates from various sources, including ERP, WMS, TMS, and third-party logistics systems, and may suffer from delays, noise, or missing values.

The impact of this limitation is that the quality of the generated plans can be compromised if the input data is incomplete or inconsistent. This can lead to inaccurate demand forecasts, suboptimal inventory allocations, and poor operational performance. To mitigate this, the system could incorporate advanced data cleaning and preprocessing techniques, such as outlier detection, imputation methods, and data normalization. Additionally, robust error handling and fallback mechanisms can be implemented to ensure that the system can still function effectively even when some data is missing or inconsistent. Integrating real-time data sources and implementing continuous data validation and quality checks can also help ensure that the input data remains reliable and up-to-date. For example, the system could be designed to automatically flag and correct inconsistencies in the data, or to use alternative data sources when primary data is unavailable.

High Uncertainty in Demand Patterns

Another limitation is the high uncertainty in demand patterns, which can be influenced by factors such as market trends, seasonality, and external disruptions. This makes it difficult to forecast accurately and plan efficiently, even with the use of advanced AI techniques.

The impact of this limitation is that the generated plans may not always align with actual demand, leading to stockouts, overstocking, and service degradation. To mitigate this, the system could incorporate more sophisticated demand forecasting models, such as those based on machine learning and deep learning, which can better capture the complex and dynamic nature of demand patterns. Additionally, implementing real-time demand sensing and adaptive planning capabilities can help the system respond more quickly and effectively to changes in demand. For example, the system could be designed to continuously monitor demand signals and adjust plans in real-time, or to use ensemble methods to combine multiple forecasting models and reduce prediction errors. This would allow the system to be more resilient to unexpected changes in demand and to maintain high levels of accuracy and reliability.

Execution Inefficiency and Adaptation Latency

A third limitation is the potential for execution inefficiency and adaptation latency. Once a plan is generated, adjustments to accommodate new constraints or changing business priorities often require restarting the planning process from scratch, resulting in significant delays and inefficiencies.

The impact of this limitation is that the system may not be able to respond quickly enough to changes in the operating environment, leading to suboptimal performance and missed opportunities. To mitigate this, the system could incorporate more flexible and adaptive planning algorithms, such as those based on reinforcement learning, which can learn from experience and adjust plans in real-time. Additionally, implementing a more modular and distributed architecture can help the system scale more effectively and handle larger and more complex planning tasks. For example, the system could be designed to break down the planning process into smaller, independent modules that can be updated and adjusted independently, or to use parallel processing to speed up the planning process. This would allow the system to be more agile and responsive, and to adapt more quickly to changes in the operating environment.

Practical Implications

Enhanced Human Productivity

The LLM-based planning assistant can significantly enhance human productivity by automating repetitive and time-consuming tasks, allowing planners to focus on higher-value activities. This can lead to more strategic and effective decision-making, ultimately improving overall operational performance.

In practical terms, this means that supply chain managers and planners can spend less time on data processing and analysis and more time on strategic planning, risk management, and innovation. For example, they can use the freed-up time to develop new strategies for demand forecasting, inventory optimization, and supply chain resilience, or to explore new business opportunities and partnerships. The 40% reduction in data processing time translates to a significant increase in operational efficiency, allowing planners to focus on more critical tasks and to make more informed and strategic decisions. This can lead to better resource allocation, improved risk management, and more innovative solutions to supply chain challenges.

Optimized Inventory Management

The LLM-based planning assistant can optimize inventory management by generating more accurate and reliable demand forecasts and inventory plans. This can lead to better alignment between supply and demand, reducing the risk of stockouts and overstocking, and improving overall inventory turnover and cash conversion cycles.

In practical terms, this means that supply chain managers can achieve higher inventory efficiency, lower holding costs, and improved service levels. For example, they can use the system to optimize inventory levels across multiple warehouses and distribution centers, ensuring that the right products are available in the right quantities at the right time. This can help reduce the risk of stockouts and overstocking, improve order fulfillment rates, and enhance customer satisfaction. The 2-3% increase in stock fulfillment rates translates to better customer satisfaction and reduced operational costs, as fewer orders are delayed or canceled due to stockouts. This can lead to higher customer loyalty and improved financial performance.

Real-Time Decision-Making

The LLM-based planning assistant can support real-time decision-making by continuously monitoring and adjusting plans based on real-time data and feedback. This can lead to more agile and responsive supply chain operations, capable of adapting to changes in demand, supply, and other external factors.

In practical terms, this means that supply chain managers can make more informed and timely decisions, even in the face of uncertainty and volatility. For example, they can use the system to dynamically adjust production schedules, inventory levels, and transportation routes in response to real-time demand signals, supplier capacity, and other operational constraints. This can help improve operational flexibility, reduce lead times, and enhance overall supply chain resilience. The 22% increase in the proportion of plans with an accuracy deviation below 5% translates to more reliable and precise decision-making, as the system can continuously refine its plans based on real-time data and feedback. This can lead to better alignment between supply and demand, improved operational efficiency, and higher levels of customer satisfaction.

Source: https://arxiv.org/abs/2509.03811