This post is co-written with Vikram Bansal from Tata Power, and Gaurav Kankaria, Omkar Dhavalikar from Oneture.
The global adoption of solar energy is rapidly increasing as organizations and individuals transition to renewable energy sources. India is on the brink of a solar energy revolution, with a national goal to empower 10 million households with rooftop solar installations by 2027. However, as the number of installations surges into the millions, a critical need has emerged: ensuring each solar panel system is properly installed and maintained. Traditional manual inspection methods—which involve physical site visits, visual assessments, and paper-based documentation—have become a significant bottleneck. They’re prone to human error, inconsistent, and can create substantial time delays. To address these challenges, Tata Power Center of Technology Excellence (CoE) collaborated with Oneture Technologies as their AI analytics partner to develop an AI-powered solar panel installation inspection solution using Amazon SageMaker AI, Amazon Bedrock and other AWS services.
 |
 |
In this post, we explore how Tata Power CoE and Oneture Technologies use AWS services to automate the inspection process end-to-end.
Challenges
As Tata Power scales up their solar panel installations, several key challenges emerge with the current process:
Time-consuming manual inspection: Traditional inspection processes require engineers to visually inspect every panel and manually document their findings. This approach is time-consuming and susceptible to human error. Engineers must carefully examine multiple aspects of the installation, from panel alignment to wiring connections, making the process lengthy and mentally taxing.
Limited scalability: The current manual inspection process cannot keep pace with the rapidly increasing volume of installations, creating a widening gap between inspection capacity and demand. As Tata Power aims to handle millions of new installations, the limitations of manual processes become increasingly apparent, potentially creating bottlenecks in installations.
Inconsistent quality standard: The deployment of multiple inspection teams across various locations affects maintaining uniform quality standards. Different teams might interpret and apply quality guidelines differently, resulting in variations in how assessments are conducted and documented. This lack of standardization makes it difficult to help achieve consistent quality across all installations.
Increasing customer escalations: Inconsistent installation quality and delays in completion results in a growing number of customer complaints and escalations. These issues directly affect customers’ experience, with customers expressing dissatisfaction over varying quality standards and extended waiting periods.
Solution overview
Implementing an AI-powered inspection system to perform more than 22 distinct checks across six different solar installation components required complex technical solutions. The inspection criteria ranged from simple visual verifications to sophisticated quality assessments requiring specialized approaches for detecting tiny defects, verifying placement accuracy, and evaluating installation completeness. The absence of a standard operating procedure (SOP) to capture images, resulting in variation in angles, lighting, object distance, and background clutter across the dataset, further complicated processes. Some criteria had abundant training data, while others had limited and imbalanced datasets, making model generalization difficult. Certain installation criteria demanded accurate distance measurements, such as verifying whether components were installed at the correct height or maintaining proper spacing between elements. Traditional computer vision models proved inadequate for these metric-based evaluations without the support of specialized sensors or depth estimation capabilities. The diversity of inspection requirements demanded a sophisticated multi-model approach, because no single computer vision model could adequately address all inspection criteria. An essential aspect lay in carefully mapping each inspection criterion to its most appropriate AI model type, ranging from object detection for component presence verification to semantic segmentation for detailed analysis, and incorporating generative AI-based reasoning for complex interpretative tasks.
To address these challenges, Tata Power CoE collaborated with Oneture to create a secure, scalable, and intelligent inspection platform using AWS services. Before technical development, the team conducted extensive field research to understand real-world installation conditions. This approach revealed key operational realities: installations occurred in tight spaces with poor lighting conditions, equipment varied significantly across sites, and image quality was often compromised by environmental factors (demonstrated in the following image). One crucial insight emerged during these field observations: certain inspection requirements, particularly measurements like the gap between inverters and walls, demanded sophisticated spatial analysis capabilities that went beyond basic object detection.
Figure 1: Example image of solar panel components
The solution includes SageMaker AI for training and inference at scale, Amazon SageMaker Ground Truth for data labeling, Amazon Bedrock for image understanding and recommendations, Amazon Rekognition for OCR, and additional AWS services. The following diagram illustrates the solution architecture.
Figure 2: Solution Architecture
Data labeling with Amazon SageMaker Ground Truth
The foundation of accurate AI-powered inspections lies in high-quality training data. To help achieve comprehensive model coverage, the team collected more than 20,000 images, capturing a wide range of real-world scenarios including varying lighting conditions and different hardware conditions. They chose SageMaker Ground Truth as their data labeling solution, using its capabilities to create custom annotation workflows and manage the labeling process efficiently. SageMaker Ground Truth proved instrumental in maintaining data quality through its human-in-the-loop workflow features. Its built-in validation mechanisms, including stratified and random sampling, helped achieve dataset robustness. Tata Power’s quality assurance experts conducted direct reviews of labeled data through the SageMaker Ground Truth interface, providing an additional layer of validation. This meticulous attention to data quality was crucial, because even minor visual misclassifications could potentially trigger incorrect warranty claims or installation rejections.
Model training with Amazon SageMaker AI
To select and train the right model, the team use the comprehensive ML capabilities of SageMaker AI to streamline both experimentation and production deployment. SageMaker AI provided an ideal environment for rapid prototyping—the team could quickly spin up Jupyter Notebook instances, which they used to evaluate various architectures for object detection, pattern classification, OCR, and spatial estimation tasks. Through this experimentation, they selected YOLOv5x6 as their primary model, which proved particularly effective at identifying small solar panel components within high-resolution installation images. The training process, initially spanning 1.5 months, was optimized through parallel experimentation and automated workflows, resulting in streamlined, 2-day iteration cycles. Through more than 100 training jobs, the team uncovered crucial insights that significantly improved model performance. They found that increasing input image resolution enhanced small object detection accuracy, while implementing pre-processing checks for image quality factors like brightness and blurriness helped maintain consistent results. Edge cases were strategically handled by generative AI models, allowing the computer vision models to focus on mainstream scenarios. By analyzing inspection criteria overlap, the team successfully consolidated the original 22 inspection points into 10 efficient models, optimizing both processing time and costs.
Amazon SageMaker Pipelines enabled rapid feedback loops from field performance data and seamless incorporation of learnings through a federated learning approach. The team could quickly adjust hyperparameters, fine-tune confidence thresholds, and evaluate model performance using metrics like F1-score and Intersection over Union (IoU), all while maintaining advanced accuracy standards. This streamlined approach transformed a complex, multi-faceted training process into an agile, production-ready solution capable of meeting stringent quality requirements at scale.
Figure 3: F1-Confidence Curve
Model inference at scale with Amazon SageMaker AI
Deploying the model presented unique requirements for Tata Power, particularly when handling high-resolution images captured in remote locations with unreliable network connectivity. While SageMaker AI real-time inference is powerful, it comes with specific limitations that didn’t align with Tata Power’s requirements, such as a 60-second timeout for endpoint invocation and a 6 MB maximum payload size. These constraints could potentially impact the processing of high-resolution inspection images and complex inference logic.
To address these operational constraints, the team implemented SageMaker AI asynchronous inference, which proved to be an ideal solution for their distributed inspection workflow. The inference ability to handle large payload sizes accommodated the high-resolution inspection images without compression, helping to ensure that no detail was lost in the analysis process. The endpoints automatically scaled based on incoming request volume, optimizing both performance and cost efficiency.
Maintaining model accuracy with SageMaker Pipelines
To help ensure sustained model performance in production, the team implemented an automated retraining system using SageMaker AI. This system continuously monitored model predictions, automatically triggering data collection when confidence scores fell below defined thresholds. This approach to model maintenance helped combat model drift and ensure that the system remained accurate as field conditions evolved. The retraining pipeline, built on SageMaker Pipelines, automated the entire process from data collection to production deployment. When new training data was collected, the pipeline orchestrated a sequence of steps: data validation, model retraining, performance evaluation in a staging environment, and finally, controlled deployment to production through continuous integration and delivery (CI/CD) integration.
OCR with Amazon Rekognition
While custom machine learning models powered much of Tata Power’s inspection platform, the CoE team recognized that some tasks could be solved more efficiently Amazon Rekognition, for example reading Ohm Meter values during inspections, as shown in the following figure.
Figure 4: Omh Meter
By integrating the OCR capabilities of Amazon Rekognition, the team avoided the time-consuming process of developing and training custom OCR models, while still achieving the advanced accuracy levels required for production use.
Enhancing the inspection process with Amazon Bedrock
While computer vision models delivered advanced accuracy for most inspection points, they had limitations with specific scenarios involving extremely small object sizes in the image, variable camera angles, and partially obscured elements. To address these limitations, The team implemented Amazon Bedrock to enhance the inspection process, focusing on six critical criteria that required additional intelligence beyond traditional computer vision. Amazon Bedrock enabled a critical pre-check phase before initiating computer vision inference operations. This pre-inference system evaluates three key image quality parameters: visibility clarity, object obstruction status, and capture angle suitability. When images fail to meet these quality benchmarks, the system automatically triggers one of two response pathways—either flagging the image for immediate recapture or routing it through specialized Generative AI reasoning processes. This intelligent pre-screening mechanism optimizes computational efficiency by preventing unnecessary inference cycles on suboptimal images, while helping to ensure high-quality input for accurate inspection results.
To close the loop, Amazon Bedrock Knowledge Bases provides real-time, contextual guidance from internal guideline documents. This automated feedback loop accelerates the inspection cycle and improves installation quality by providing instant, actionable recommendations at the point of inspection.
The mobile app
The mobile app provides an intuitive interface designed specifically for on-site use, so that engineers can efficiently complete installation inspections through a streamlined workflow. With this app, field engineers can capture installation photos, receive immediate analysis results, and validate AI findings all through a single interface
Results and impact
The implementation of the AI-powered automated inspection tool delivered measurable improvements across Tata Power’s solar installation operations.
- The solution achieves more than 90% AI/ML accuracy across most of the points with object detection precision of 95%, enabling near real-time feedback to channel partners instead of delayed offline reviews.
- Automated quality checks now instantly verify most installations, significantly reducing manual inspection needs. AI model training continues to improve accuracy in detecting missing checkpoints.
- Re-inspection rates have dropped by more than 80%. These efficiency gains led to faster site handovers, directly improving customer satisfaction metrics.
- The automated system’s ability to provide immediate feedback enhanced channel partner productivity and satisfaction, creating a more streamlined installation process from initial setup to final customer handover.
Conclusion
In this post, we explained how Tata Power CoE, Oneture Technologies, and AWS transformed traditional manual inspection processes into efficient, AI-powered solutions. By using Amazon SageMaker AI, Amazon Bedrock, and Amazon Rekognition, the team successfully automated solar panel installation inspections, achieving more than 90% accuracy while cutting re-inspection rates by 80%.See the following resources to learn more:
- Visit the AWS Community to discover how our builder communities are using Amazon SageMaker AI and Amazon Bedrock in their solutions.
- Learn more about Amazon SageMaker AI
- Learn more about Amazon Bedrock
