Data Science

  Unlocking the True Cost of Generative AI As organizations increasingly integrate generative AI into their operations, understanding the associated costs is crucial for effective budgeting and resource allocation. Here's a comprehensive breakdown of the various cost types involved in building and maintaining AI systems: GenAI Tools & Platform Access Costs Prompt Engineering Costs Inference Costs Fine-Tuning Costs Infrastructure Costs Data Management Costs Operational Costs AI Regulations Compliance Costs Talent Costs Change Management Costs By strategically managing these cost types, you can maximize the investment in generative AI and drive innovation forward. What are your thoughts on these cost factors? How are you addressing them?

 LLM Evaluation Guide Large Language Model (LLM) is the industry buzz word in recent years. It can understand human language and plays crucial roles in applications like chatbots, translations, and content creation. Evaluating LLMs is vital to ensure they produce accurate, relevant, and reliable outputs while minimizing biases and errors. Effective evaluation helps identify the strengths and weaknesses of these models, ensuring they perform well in real-world scenarios. Key metrics include BLEU and ROUGE for text quality, BERTScore and MoverScore for semantic similarity, and QuestEval for relevance and completeness. Proper evaluation guarantees that LLMs meet high standards and user expectations. Here are few dimensions on which LLMs can be evaluated. - Evaluating Generated Text Quality - Evaluating Semantic Similarity - Evaluating Factual Consistency - Evaluating Relevance and Completeness - Detecting Hallucinations - Evaluating User Preferences - No References Available What othe...

 Python Turtle Package “Turtle” is a python feature like a drawing board, which lets you command a turtle to draw all over it! You can use functions like turtle.forward(...) and turtle.left(...) which can move the turtle around. It was part of the original Logo programming language developed by Wally Feurzeig, Seymour Papert and Cynthia Solomon in 1967 Short Tutorial

Machine Learning Part 11 - Feature Selection Feature Selection

 Data Science Interview Questions

 Time Series Forecasting - Basics PDF:

 Machine Learning - Books & Materials Why does it matter? With industry 4.0 transformation domain knowledge along with machine learning skills will present a killer combination What is Machine Learning? If you search internet you will end up getting hundreds of courses, mathematical theorems or hi-fi corporate jargons and stories about artificial intelligence, fascinating data science and dream jobs of future Say you want to get a phone. Years back you might have visited a store compared the specifications, price, color etc. and spent hours before making a decision. However now while enjoying the comfort of your home you give some inputs like budget, OS, camera etc. and you see the results on internet. The field of study with one goal of turning data to intelligent action or predictions is known as “Machine Learning” Free ML Books:- ML for beginners Machine Learning For Dummies ML Intro ML By Andrew NG Math for ML Pandas_1 Introduction to Machine Learning with Python Machine le...

Machine Learning - Part 9 Advanced Regression PDF: Video:  

 Deep Learning - Part 1 What will you learn? What is Artificial Neuron? How does it work What is math behind an articifical neuron? PDF: Video:

Deep Learning Books & Materials   Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks The scalability of neural networks is key factor indicating the importance of Deep Learning (with more data and larger models, that in turn require more computation to train) In addition to scalability, another often cited benefit of deep learning models is their ability to perform automatic feature extraction from raw data, also called feature learning. Books & Materials: Neural Networks Intro Neural_Network and Deep_Learning Deep Learning Methods and Applications Deeplearning_notes Dive into Deep Learning DL Notes from Andrew Ng Course Machine Learning Neural and Statistical Classification Deep Learning Tutorial with Python AutoEncoders CNN_Cheatsheet RNN_cheatsheet Keras_Cheat_Sheet_Python_1 Keras_Cheat_Sheet_Python_2 Google Drive Link

 Data Science Books & Materials Image Source: UC Berkeley Data science: Data science is the practice of analysing raw data through a wide variety of disciplines and expertise areas to produce a holistic view and predict uncertainity. What can data science be used for? Anomaly detection (transaction fraud, disease, faulty sensor etc.) Classifications (spam detection) Forecasting (sales, revenue and customer retention) Pattern detection (weather patterns, financial market patterns, etc.) Recognition (facial, voice, text, etc.) Recommendations (recommend a movie, hotel etc.) Below are some of the useful / free books & materials that I collected from various online sources. it will be useful if you are looking to transition into data science field. Introduction to Data Science Data Science from Scratch Data Science for Business - Foster Provost & Tom Fawcett Data Analytics Data Cleaning Data Engineering Data Science Statistics Cheatsheet Data Science  - Algorithms-Ha...

 Data Storytelling - Part 3 Data Storytelling =  (Context + Audience + Visual) * Narrative Video Tutorial

 Statistics for Data Science - All in one place The purpose of this is to provide an list of statistics materials needed to start your data science journey. Statistics can be a powerful tool when performing the art of Data Science. It provides deeper insights about data. Below I have compiled some list of books/materials on statitics for data science Practical Statistics for Data Science All about Statistics and Probability How to Lie with Statistics Statistics for EDA in Python Probability Cheatsheet Cartoon Guide to Statistics My Statistics Guide 1 What to learn? Where to Look? Advanced Statistics Learn R for Applied Statistics_ With Data Visualizations, Regressions, and Statistics Business Statistics Statistics for people who hate statistics Statistical Analysis with Excel For Dummies 3rd Edition Introduction to Python for Econometrics, Statistics and Data Analysis Statistical and Machine-Learning - Techniques for Better Predictions Learner's Guide to Big Numbers Statistics and ...

All in One Cheat Sheets - Python Here is compiled list of all cheat sheets related to Python. All these cheat sheets were collected from various sources and compiled in one place to benefit all Note: This page will be updated with new cheat sheets and guides wehenever available Python Beginners Cheat Sheets All Lists Dictionaries If-While Loop Functions Classes File and Exception Handling Code Testing Libraries Numpy Pandas_1 Pandas_2 Pygame Matplotlib Seaborn Plotly Bokeh Django Scikit Scikit_1 Scikit_2 SciPy Keras_1 Keras_2 Python for DS PySpark SQL PySpark RDD Spacy Jupyter Notebook NumPy_SciPy_Pandas_Quandl Data Mining & Machine Learning Cheat Sheet Guides: Visualization Guide_1 Visualization Guide_2 Python Loops Guide Python Data Structures Guide DS for Business Leaders P.S. Google Drive Link

 Machine Learning Part VIII Polynomial Regression Sample Code Video Tutorial

 Auto ML - PDF Table Extraction Image: excalibur-py.readthedocs.io Extracting tables from PDFs is not easy. Simple copy and paste from a PDF don't preserve table structure. Hence automatically detecting the structure and preserving the format is critical. Machine Learning come to rescue here as well. Let us see some of the Python libraries available for this task Excalibur: It is a web interface to extract tabular data from PDFs. It is powered by Camelot. It only works with text-based PDFs and not scanned documents. Installation Guide Tutorial PDF Table Extraction: It is a parser to extract the table in PDF document with RetinaNet Github/Installation Guide Tutorial Camelot: It is a Python library and a command-line tool that makes it easy for anyone to extract data tables trapped inside PDF file Installation Guide Tutorial Github Tabula: It is a free tool for extracting data from PDF files into CSV and Excel files. Tabula only works on text-based PDFs, not scanned docu...

  Essential Statistics for Data Science The goal of inferential statistics is to use the sample to learn about the population The steps involved here are Sampling Hypothesis Testing Inference Sample typically is selected in a manner that allows it to be an unbiased representation of the entire population Sampling the entire population is nearly impossible and Central Limit Theorem helps solve this problem Some of the business cases are Estimate the effectiveness of a medicine or vaccine Estimate the expected consumers for a new product/service

  Importance of Data Preparation & Common Methods   Data preparation is the various methods and steps taken to transform the data such that it is suitable for ML algorithms. Here quality precedes over quantity. Hence it is essential to transform the raw data into to a more informative format for a better modelling task. Below are the most commonly followed data preparation tasks in data science industry   Data Cleaning Feature Engineering Data Transformation Feature Extraction   Data Cleaning: Data cleaning is the process of detecting and also treating inaccurate data points from a data set. This is the most time consuming and critical part of ML. It simply goes "garbage in = garbage out". Below are some of the methods to clean data Find the number / percentage (both) of missing rows or null values (NAN) & treat it Identify the duplicate rows within the dataset & treat it Detect the outliers using statistics & treat it using doma...

 Machine Learning Part VII -  KNN Classifier Video Link Sample Code

    Machine Learning Part VI -  Logistic Regression Video Link Sample Code