Data science and classification

Updated 27 Oct 2026

•
Data science and task types
- •
  Data science -> uses data to describe patterns, predict outcomes, explain evidence, or support decisions.
  - •
    Descriptive analytics -> what happened, using historical data.
  - •
    Diagnostic analytics -> investigate why it happened.
  - •
    Predictive analytics -> forecast what may happen.
  - •
    Prescriptive analytics
    
    •
    Task -> recommend what should be done.
    
    •
    domain knowledge: remains necessary.
- •
  before modeling, inspect: units, missingness, distributions, duplicates, outliers, labels, collection conditions.
  - •
    Survivorship-bias example
    
    •
    observed cases: damage on returning WWII aircraft.
    
    •
    missing cases: aircraft that did not return.
    
    •
    missing cases can change the conclusion.
  - •
    Scale correction
    
    •
    Elapsed age -> ratio-scale, not interval-scale.
    
    •
    Category codes -> numeric codes remain categories.
- •
  Outlier -> may be an error, rare valid case, or important event.
- •
  Z-score -> $z=\frac{x-\mu}{\sigma}$ .
  - •
    Sample standardization
    
    •
    Formula -> $z=(x-\bar x)/s$ .
    
    •
    Sample condition -> $s>0$ .
    
    •
    Population condition -> $\sigma>0$ .
  - •
    Screening caution
    
    •
    $|z|>3$ is a heuristic, not proof of error.
    
    •
    distribution shape: skewness and extreme values affect mean/standard-deviation screening.
    
    •
    Distribution assumption -> interpreting $z$ as a tail probability assumes a distribution.
- •
  Supervised learning -> learns from labeled examples.
  - •
    Classification -> predicts categories.
  - •
    Regression -> predicts numeric values
- •
  Unsupervised learning -> finds structure without target labels, such as clustering
- •
  model decision factors: data quality, evaluation, error costs, thresholds, feedback.
- •
  Data mining and data science
  - •
    Data mining -> discovers useful patterns in datasets.
  - •
    Data science also includes framing questions, interpreting results, and supporting decisions.
  - •
    the four analytics types above describe analytical purposes, not four universal types of decision.
- •
  AI and machine learning
  - •
    machine learning can build models used in AI-based systems.
  - •
    not all data science requires machine learning.
  - •
    not all AI requires machine learning.
- •
  - •
    test
•
CRISP-DM: from business need to use
- •
  CRISP-DM -> Cross-Industry Standard Process for Data Mining.
  - •
    six connected phases, with iteration between them rather than a one-way checklist
- •
  Phases
  - •
  1. 1.
    Business understanding
    
    1.
    identify the problem, users' needs, and system needs.
    
    2.
    translate the business problem into an analytics task and success criteria.
    
    3.
    specify the data needed.
    
    4.
    plan responsibilities, resources, and risks.
  2. 2.
    Data understanding
    
    1.
    collect initial data, inspect it, and validate quality.
    
    2.
    check whether it can support the proposed task.
    
    3.
    revise the problem or collection plan when necessary.
  3. 3.
    Data preparation
    
    1.
    select relevant data and clean errors or missing values.
    
    2.
    construct useful features and integrate sources.
    
    3.
    fit learned preprocessing only on training data to avoid evaluation leakage
  4. 4.
    Modeling
    
    1.
    choose suitable analytical or ML methods.
    
    2.
    define the comparison/search strategy.
    
    3.
    plan training and evaluation splits.
    
    4.
    fit candidate models and tune using validation evidence.
    
    5.
    Training -> training data + learning method -> fitted model.
    
    6.
    Evaluation data -> checks performance rather than supplying the final answer in advance.
  5. 5.
    Evaluation
    
    1.
    Compare performance with business targets, baselines, and error costs.
    
    2.
    Review data suitability, preparation, and assumptions.
    
    3.
    Check whether the result solves the original problem.
    
    4.
    The best score among tested models does not automatically mean the best business choice.
  6. 6.
    Deployment
    
    1.
    Put results into use as a report, decision procedure, or operational model.
    
    2.
    Plan monitoring, ownership, maintenance, and feedback as conditions change.
- •
  information-systems-polban lecture grouping
  - •
    phases 1-2 -> needs analysis.
  - •
    phases 3-5 -> model development.
  - •
    deployment -> model use.
- •
  Related data workflows
  - •
    Knowledge discovery
    
    •
    selection: data -> target data.
    
    •
    preprocessing: target data -> cleaned data.
    
    •
    transformation: cleaned data -> transformed data.
    
    •
    data mining: transformed data -> patterns.
    
    •
    interpretation: make sense of the patterns.
  - •
    ETL -> extract, transform, load.
    
    1.
    read from sources.
    
    2.
    convert or reshape data.
    
    3.
    write to a destination.
    
    •
    ETL supports data preparation, not the full CRISP-DM lifecycle.
- •
  Tools -> Weka, KNIME, and R.
  - •
    the method and evaluation plan matter more than a particular tool.
- •
  Task families
  - •
    Classification
  - •
    Regression
  - •
    Clustering
  - •
    Association rules
- •
•
Classifiers
- •
  Decision tree
  - •
    Internal nodes -> feature tests.
  - •
    Leaves -> predictions.
  - •
    Entropy -> $H(S)=-\sum_i p_i\log_2(p_i)$ .
  - •
    Information gain -> reduction in entropy after a split.
    
    •
    Weighted split formula -> $IG(Y,X)=H(Y)-\sum_vP(X=v)H(Y\mid X=v)$ .
    
    •
    Entropy convention
    
    •
    Zero-probability term -> $0\log_2 0=0$ .
    
    •
    Pure node: entropy $0$ .
    
    •
    Fourteen-row example
    
    •
    Class counts -> $10$ yes/ $4$ no.
    
    •
    Entropy -> $H\approx0.863121$ bits.
    
    •
    Outlook gain -> $0.258521$ .
    
    •
    Temperature gain -> $0.183851$ .
    
    •
    Humidity gain -> $0.370507$ .
    
    •
    Windy gain -> $0.005978$ .
    
    •
    Humidity has the highest information gain.
    
    •
    Gain ratio
    
    •
    Formula -> $\mathrm{GainRatio}=IG/\mathrm{SplitInfo}$ .
    
    •
    split information: $\mathrm{SplitInfo}=-\sum_vP(X=v)\log_2P(X=v)$ .
    
    •
    condition: denominator must be nonzero.
    
    •
    purpose: addresses preference for many-branch attributes.
    
    •
    Algorithms
    
    •
    ID3 -> information gain for classification.
    
    •
    C4.5 -> gain-ratio selection and pruning.
    
    •
    CART -> binary classification or regression trees.
    
    •
    information gain alone does not demonstrate complete C4.5.
  - •
    overfitting control -> limit depth or prune the tree.
- •
  Linear classifier -> uses a linear decision boundary.
- •
  Nonlinear classifier -> can form a more complex decision boundary.
- •
  K-nearest neighbors (KNN)
  - •
    predicts from nearby labeled examples.
  - •
    scale features when units or ranges differ.
  - •
    Distance -> For $p$ features: $d(x,y)=\sqrt{\sum_{j=1}^{p}(x_j-y_j)^2}$ .
  - •
    Parcel example: $P5=(6\text{ km},4\text{ traffic points})$ .
    
    •
    Other training point
    
    •
    Coordinates -> $P1=(4,2)$ .
    
    •
    class: on time
    
    •
    Distance -> $\sqrt8\approx2.83$ .
    
    •
    selection: outside the three nearest neighbors
    
    •
    Neighbor P4
    
    •
    Coordinates -> $P4=(7,4)$ .
    
    •
    class: late.
    
    •
    Distance -> $1.00$ .
    
    •
    Neighbor P3
    
    •
    Coordinates -> $P3=(5,3)$ .
    
    •
    class: on time.
    
    •
    Distance -> about $1.41$ .
    
    •
    Neighbor P2
    
    •
    Coordinates -> $P2=(8,5)$ .
    
    •
    class: late.
    
    •
    Distance -> about $2.24$ .
    
    •
    Majority vote
    
    •
    prediction: with $K=3$ , predict late
    
    •
    the prediction does not guarantee the outcome.
- •
  Naive Bayes
  - •
    theorem: applies Bayes' theorem
  - •
    Assumption -> conditional feature independence.
    
    •
    Prediction
    
    •
    Formula -> $\hat y=\arg\max_yP(y)\prod_jP(x_j\mid y)$ .
    
    •
    features are independent given the class, not unconditionally independent.
    
    •
    Variants
    
    •
    Gaussian -> continuous feature distributions.
    
    •
    Multinomial -> commonly for counts.
    
    •
    Bernoulli -> binary features.
- •
  Logistic regression -> estimates class probabilities.
  - •
    Binary model
    
    •
    Linear predictor -> $z=w^Tx+b$ .
    
    •
    Class probability -> $p=P(Y=1\mid x)=1/(1+e^{-z})$ .
    
    •
    log-odds: $\log[p/(1-p)]=w^Tx+b$ .
  - •
    Threshold
    
    •
    decision: predict class 1 when probability exceeds the chosen threshold.
    
    •
    0.5 is common, not mandatory.
    
    •
    choose with error costs in mind.
  - •
    Checkout example
    
    •
    Supplied weights -> $w_1=0.05$ , $w_2=0.8$ , $b=-2$ .
    
    •
    inputs: duration $10$ , items $3$ .
    
    •
    Linear predictor -> $z=0.9$ .
    
    •
    Probability -> $p\approx0.7109495$ .
    
    •
    prediction: checkout at threshold $0.5$ .
    
    •
    fitting caution: this evaluates supplied parameters, not a demonstrated fit to the five observations.
- •
  Support vector machine -> seeks a large-margin boundary.
- •
  2D KNN sketch: plot $P5$ and its three nearest neighbors, label distances/classes, and mark the predicted class.
•

Linked references 2

◌ Explore connections in Graph view

Data science and classification

Data science and task types

CRISP-DM: from business need to use

Classifiers

Linked references 2