동선생

While training machine learning models, it is necessary to preprocess missing values because it might greatly affect the performance of the model. Accordingly, there are various methods to deal with those missing values. Types of Missing Values:Missing completely at random (MCAR)Case when missing values occur independently of other variables, completely random in other words.No correlation with ..

First competition following Yuhan Lee's kaggle curriculum. Binary classification competition using tabular data.  First KernelEDA(Exploratory Data Analysis) written on Titanic competition by Yuhan himself.visualization tool used: matplotlib, seaborn, plotlydata analysis tool used: pandas, numpymachine learning tool: sklearncode: https://github.com/LDYWINNER/TIML/blob/main/kaggle-roadmap/binary-c..

My goal is to be a ranker in Kaggle community and I want to be somekind of running mate with readers of my blog. So I am planning to write some posts like this that records how I am studying.  As of 2024.10.25, I have some background knowledge about data science because I took some undergraduate courses and studied for about 1 or 2 month before. So, I just started to compete in kaggle instead of..

Loss FunctionMachine learning algorithms find the patterns of the given dataset by themselves. For example, let's think about a situation that we should predict whether it will rain tomorrow or not from humidity data. 1.5 * x + 0.1 = y (raining if y > 1) If we can simply express the relationship between the probability of rain and the humidity like this, "pattern" of this data is 1.5(weight) and..

What is Ensemble method?In real-world problems, it is obvious that we can end up in better conclusions if we ask for others' opinion and get advice. It works the same with making machine learning models: if we take a look at the results of various solutions and pick the best among them or deciding by majority, we can make better conclusion. Models trained for drawing conlcusions are called "ense..

Overfitting and UnderfittingBefore we talk about what cross validation method is, we should first think about overfitting and underfitting in terms of machine learning. In general ml problems, we utilize explanatory variables from train data to predict the target variable in the test data. However, test data would not have target variable which makes difficult for us to validate the performance ..

What is Curse of DimensionalitySolving machine learning problems, we often face problems on train data having excessive number of features. Due to this problem, training speed gets slower and improving model performance proves much more challenging in this case.This kind of problem is called "Curse of Dimensionality". Obviously, there are methods we can deal with this curse: simply reducing the ..

What is Time-series Data?To solve a certain problem, we should utilize various types of data to extract key information from it and make the machine-learning model learn to perform better prediction. Time-series data, which is one of common type of data, is a successive sequence of data such as observation of a car's speed or fluctuation of stock prices.  The key difference between time-series d..

AutoEncoder란?AutoEncoder는 비지도 학습 신경망 모델로, 레이블되어 있지 않은 훈련 데이터를 사용하여 압축(인코딩)하고 다시 복원(디코딩)하는 과정을 통해 효율적인 데이터 표현을 학습합니다. 또한, unsupervised learning(비지도 학습)을 supervised learning(지도 학습) 문제로 바꿔서 해결한다고 표현할 수 있습니다.일반적으로 지도 학습은 입력 데이터 X와 타겟 데이터 y를 모두 사용하여 학습합니다. 반면, 비지도 학습은 입력 데이터 X만 사용하여 모델을 학습시킵니다.AutoEncoder의 특징은 입력 데이터 X만 주어지지만, 타겟 데이터도 X라는 점입니다: 원본 데이터 X를 입력으로 받고 재구성된 데이터 X'을 출력합니다. AutoEncoder의 목표는 X..

해당 포스트는 auto encoder를 학습하던 중 지도 학습과 비지도 학습에 대한 개념을 학습하기 위해 작성되었습니다. 지도 학습(Supervised Learning)지도 학습이란 머신러닝 모델이 데이터를 학습할 때, 학습하는 데이터가 레이블(label)되어 있는 경우를 말합니다.  지도 학습의 종류에는 크게 회귀(Regression)와 분류(classification)이 있습니다. 회귀는 훈련 데이터와 이에 매칭되는 타겟 변수의 값들(레이블)을 학습하여, 새로운 입력 데이터에 대한 출력 값을 예측하는 기법입니다.분류는 말그대로 훈련 데이터와 이에 매칭되는 레이블을 학습하여, 새로운 입력 데이터의 범주형(categorical) 클래스를 예측하는 기법입니다.비지도 학습(Unsupervised Learni..