Deepankar Sharma

Text Summarization

The Text-Summarizer project is a comprehensive and efficient solution designed to summarize long passages of text into concise and informative summaries. The project follows a structured training pipeline with five main stages: Data Ingestion, Data Validation, Data Transformation, Model Trainer, and Model Evaluation.

The Data Ingestion stage is responsible for ingesting raw data required for model training, while the Data Validation stage ensures the integrity and correctness of the ingested data. In the Data Transformation stage, the raw data is preprocessed and transformed into a suitable format for model training. The Model Trainer stage involves training the summarization model on the preprocessed data, using the Pegasus model checkpoint.

Finally, in the Model Evaluation stage, the trained model is evaluated using evaluation metrics to assess its performance. The project also includes an app.py file that serves as the application module, using FastAPI to create a web-based interface for users to access the summarization functionality. The app allows users to trigger model training through the `/train` route and input text for summarization through the `/predict` route.

The configuration file (config.yaml) organizes essential settings and paths required for different stages of the pipeline, making the workflow efficient and easy to customize. The Text-Summarizer project streamlines the process of text summarization, providing valuable summaries for various use cases, including news articles, research papers, and legal documents. With its systematic approach and web-based interface, this project proves to be a valuable tool for efficient text processing and information extraction.