{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# CSC 3105 Project"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "cda961ffb493d00c"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Load and Clean the Data\n",
    "\n",
    "This code block performs the following operations:\n",
    "\n",
    "1. Imports necessary libraries for data handling and cleaning.\n",
    "2. Defines a function `load_data` to load the data from a given directory into a pandas dataframe.\n",
    "3. Defines a function `clean_data` to clean the loaded data. The cleaning process includes:\n",
    "   - Handling missing values by dropping them.\n",
    "   - Removing duplicate rows.\n",
    "   - Converting the 'NLOS' column to integer data type.\n",
    "   - Normalizing the 'Measured range (time of flight)' column.\n",
    "   - Creating new features 'FP_SUM' and 'SNR'.\n",
    "   - One-hot encoding categorical features.\n",
    "   - Performing feature extraction on 'CIR' columns.\n",
    "   - Dropping the original 'CIR' columns.\n",
    "   - Checking for columns with only one unique value and dropping them.\n",
    "4. Checks if a pickle file with the cleaned data exists. If it does, it loads the data from the file. If it doesn't, it loads and cleans the data using the defined functions.\n",
    "5. Prints the first few rows of the cleaned data and its column headers."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "73fe8802e95a784f"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import accuracy_score, classification_report\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "import pickle\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.metrics import roc_curve, auc\n",
    "import tensorflow as tf\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pywt\n",
    "from skimage import restoration\n",
    "from tensorflow.keras.utils import to_categorical\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:21.260710Z",
     "start_time": "2024-03-20T11:36:21.256027Z"
    }
   },
   "id": "2aa3c6c09e8645d1",
   "execution_count": 70
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# Define the directory where the dataset is located\n",
    "DATASET_DIR = './UWB-LOS-NLOS-Data-Set/dataset'\n",
    "\n",
    "def load_data(dataset_dir):\n",
    "    # Load the data\n",
    "    # file_paths = [os.path.join(dirpath, file) for dirpath, _, filenames in os.walk(dataset_dir) for file in filenames if 'uwb_dataset_part7.csv' not in file]\n",
    "    file_paths = [os.path.join(dirpath, file) for dirpath, _, filenames in os.walk(dataset_dir) for file in filenames]\n",
    "    data = pd.concat((pd.read_csv(file_path) for file_path in file_paths))\n",
    "    print(f\"Original data shape: {data.shape}\")\n",
    "    return data\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:21.918674Z",
     "start_time": "2024-03-20T11:36:21.913408Z"
    }
   },
   "id": "7bcd7cfc8dd11cbb",
   "execution_count": 71
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "def stat_analysis_and_plots(data):\n",
    "    # Statistical Analysis\n",
    "    print(\"Statistical Analysis:\")\n",
    "    print(data.describe())\n",
    "\n",
    "    # Plot Boxplot to check for outliers for the first 15 columns\n",
    "    print(\"Boxplot of the first 15 columns:\")\n",
    "    fig, axs = plt.subplots(15,1,dpi=95, figsize=(7,17))\n",
    "    for i, col in enumerate(data.columns[:15]):\n",
    "        axs[i].boxplot(data[col], vert=False)\n",
    "        axs[i].set_ylabel(col)\n",
    "    plt.show()\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:21.998580Z",
     "start_time": "2024-03-20T11:36:21.993122Z"
    }
   },
   "id": "9e0b1ed6f23a17cf",
   "execution_count": 72
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Channel Impulse Response (CIR) Graphs\n",
    "\n",
    "This code block defines a function `cir_graphs` to plot the Channel Impulse Response (CIR) for Line of Sight (LOS) and Non-Line of Sight (NLOS) data. The CIR is a sequence of values representing the channel response to a single impulse. It is used to characterize the channel in wireless communication systems."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "1dd92fe7b6881ea6"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "def cir_graphs(data):\n",
    "    # Separate the data into LOS and NLOS\n",
    "    data_los = data[data['NLOS'] == 0]\n",
    "    data_nlos = data[data['NLOS'] == 1]\n",
    "\n",
    "    # Extract the CIR columns\n",
    "    cir_columns = [col for col in data.columns if 'CIR' in col]\n",
    "    data_los_cir = data_los[cir_columns]\n",
    "    data_nlos_cir = data_nlos[cir_columns]\n",
    "\n",
    "    # Calculate the magnitude and time for each CIR column\n",
    "    time_los = np.arange(len(data_los_cir.columns))\n",
    "    magnitude_los = np.linalg.norm(data_los_cir.values, axis=0)\n",
    "\n",
    "    time_nlos = np.arange(len(data_nlos_cir.columns))\n",
    "    magnitude_nlos = np.linalg.norm(data_nlos_cir.values, axis=0)\n",
    "\n",
    "    # Plot the magnitude vs time for LOS\n",
    "    plt.figure(figsize=(20, 10), dpi=300)  # Increase figure size and DPI\n",
    "    plt.plot(time_los, magnitude_los, linewidth=2)  # Increase line width\n",
    "    plt.title('Magnitude vs Time for LOS')\n",
    "    plt.xlabel('Time (ns)')\n",
    "    plt.ylabel('Magnitude')\n",
    "    plt.xlim([600, max(time_los)])  # Set x-axis limits\n",
    "    # plt.ylim([0, 2e6])\n",
    "    plt.show()\n",
    "\n",
    "    # Plot the magnitude vs time for NLOS\n",
    "    plt.figure(figsize=(20, 10), dpi=300)  # Increase figure size and DPI\n",
    "    plt.plot(time_nlos, magnitude_nlos, linewidth=2)  # Increase line width\n",
    "    plt.title('Magnitude vs Time for NLOS')\n",
    "    plt.xlabel('Time (ns)')\n",
    "    plt.ylabel('Magnitude')\n",
    "    plt.xlim([600, max(time_los)])  # Set x-axis limits\n",
    "    # plt.ylim([0, 2e6])\n",
    "    plt.show()\n",
    "    "
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.090025Z",
     "start_time": "2024-03-20T11:36:22.080262Z"
    }
   },
   "id": "308d64639b199bc7",
   "execution_count": 73
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "def calculate_total_distance(data):\n",
    "    # Speed of light in meters per nanosecond\n",
    "    speed_of_light_ns = 0.299792458\n",
    "\n",
    "    # Extract the CIR columns\n",
    "    cir_columns = [col for col in data.columns if 'CIR' in col]\n",
    "\n",
    "    # Calculate the total distance for each row\n",
    "    data['Total_Distance'] = data[cir_columns].abs().sum(axis=1) * speed_of_light_ns\n",
    "\n",
    "    return data"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.129821Z",
     "start_time": "2024-03-20T11:36:22.125340Z"
    }
   },
   "id": "80cfcfac265d9357",
   "execution_count": 74
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Signal to Noise Ratio (SNR) Graph\n",
    "\n",
    "This code block defines a function `snr_graph` to plot the Signal to Noise Ratio (SNR) for Line of Sight (LOS) and Non-Line of Sight (NLOS) data. The SNR is calculated as the ratio of the 'CIR_PWR' to the 'STDEV_NOISE' for each data point."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "bfd97fbe797a7067"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "\n",
    "def snr_graph(data):\n",
    "    # Separate the data into LOS and NLOS\n",
    "    data_los = data[data['NLOS'] == 0]\n",
    "    data_nlos = data[data['NLOS'] == 1]\n",
    "\n",
    "    # Extract the SNR values\n",
    "    snr_los = data_los['SNR']\n",
    "    snr_nlos = data_nlos['SNR']\n",
    "\n",
    "    # Create a new figure\n",
    "    plt.figure(figsize=(10, 5))\n",
    "\n",
    "    # Plot SNR for LOS\n",
    "    plt.plot(snr_los, label='LOS')\n",
    "\n",
    "    # Plot SNR for NLOS\n",
    "    plt.plot(snr_nlos, color='red', label='NLOS')\n",
    "\n",
    "    # Set title and labels\n",
    "    plt.title('SNR for LOS and NLOS')\n",
    "    plt.xlabel('Index')\n",
    "    plt.ylabel('SNR')\n",
    "\n",
    "    # Add a legend\n",
    "    plt.legend()\n",
    "\n",
    "    # Show the plot\n",
    "    plt.show()\n",
    "    "
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.213753Z",
     "start_time": "2024-03-20T11:36:22.208758Z"
    }
   },
   "id": "4afc8d71b3271351",
   "execution_count": 75
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from scipy.stats import norm\n",
    "\n",
    "def plot_histogram(data, feature):\n",
    "    \"\"\"\n",
    "    Function to plot a histogram of a given feature in the data for 'NLOS' and 'LOS'.\n",
    "\n",
    "    Parameters:\n",
    "    data (pd.DataFrame): The data.\n",
    "    feature (str): The name of the feature to plot.\n",
    "\n",
    "    Returns:\n",
    "    None\n",
    "    \"\"\"\n",
    "    # Check if the feature exists in the data\n",
    "    if feature not in data.columns:\n",
    "        print(f\"The feature '{feature}' does not exist in the data.\")\n",
    "        return\n",
    "\n",
    "    # Separate the data into 'NLOS' and 'LOS'\n",
    "    data_nlos = data[data['NLOS'] == 1]\n",
    "    data_los = data[data['NLOS'] == 0]\n",
    "\n",
    "    # Create a figure with two subplots side by side\n",
    "    fig, axs = plt.subplots(1, 2, figsize=(10, 5))\n",
    "\n",
    "    # Plot the histogram for 'NLOS'\n",
    "    axs[0].hist(data_nlos[feature], bins=30, edgecolor='black')\n",
    "    axs[0].set_title(f'Histogram of {feature} for NLOS')\n",
    "    axs[0].set_xlabel(feature)\n",
    "    axs[0].set_ylabel('Frequency')\n",
    "\n",
    "    # Plot the histogram for 'LOS'\n",
    "    axs[1].hist(data_los[feature], bins=30, edgecolor='black')\n",
    "    axs[1].set_title(f'Histogram of {feature} for LOS')\n",
    "    axs[1].set_xlabel(feature)\n",
    "    axs[1].set_ylabel('Frequency')\n",
    "\n",
    "    # Display the plots\n",
    "    plt.tight_layout()\n",
    "    plt.show()\n",
    "\n",
    "# Usage:\n",
    "# plot_histogram(data, 'First_Path_Power_Level')"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.268611Z",
     "start_time": "2024-03-20T11:36:22.260730Z"
    }
   },
   "id": "22025d6c8281fc09",
   "execution_count": 76
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "def plot_features(data, labels, feature1, feature2):\n",
    "    reds = labels == 1\n",
    "    blacks = labels == 0\n",
    "    plt.scatter(data[reds][feature1], data[reds][feature2], c=\"red\", s=20, edgecolor='k')\n",
    "    plt.scatter(data[blacks][feature1], data[blacks][feature2], c=\"yellow\", s=20, edgecolor='k')\n",
    "    plt.xlabel(feature1)\n",
    "    plt.ylabel(feature2)\n",
    "    plt.title(f\"Plot of data: {feature1} versus {feature2}\")\n",
    "    plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.354769Z",
     "start_time": "2024-03-20T11:36:22.349304Z"
    }
   },
   "id": "ac4db13fed3f9916",
   "execution_count": 77
  },
  {
   "cell_type": "markdown",
   "source": [
    "## denoise_cir Function\n",
    "\n",
    "The `denoise_cir` function uses the Discrete Wavelet Transform (DWT) to denoise the Channel Impulse Response (CIR) values. The DWT is a linear transformation that operates on a data vector whose length is an integer power of two, transforming it into a numerically different vector of the same length. The DWT of a signal `x` is calculated as follows:\n",
    "\n",
    "1. **Wavelet Decomposition:**\n",
    "\n",
    "    The input signal `x` is passed through two complementary filters and emerges as two signals. The filter outputs are decimated by 2 (down-sampled) to get the approximation coefficients (cA) and detail coefficients (cD).\n",
    "\n",
    "    The approximation coefficients represent the high-scale, low-frequency component of the signal, while the detail coefficients represent the low-scale, high-frequency component.\n",
    "\n",
    "2. **Thresholding:**\n",
    "\n",
    "    The detail coefficients are thresholded to remove noise. The thresholding function `T` applied to the detail coefficients `x` is defined as:\n",
    "$$\n",
    "T(x) = x * I(|x| > \\text{{value}}) \\quad \\text{{for 'hard' thresholding}}\n",
    "$$\n",
    "\n",
    "$$\n",
    "T(x) = \\text{{sign}}(x)(|x| - \\text{{value}})_+ \\quad \\text{{for 'soft' thresholding}}\n",
    "$$\n",
    "\n",
    "where $I$ is the indicator function that is one if the argument is true and zero otherwise, $\\text{{value}}$ is the threshold value, and $(x)_+$ equals $x$ if $x > 0$ and zero otherwise.\n",
    "\n",
    "3. **Wavelet Reconstruction:**\n",
    "\n",
    "    The original signal is reconstructed from the approximation and detail coefficients."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "69413268ac5b549d"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "def denoise_cir(cir_values, wavelet='db1', level=1):\n",
    "    # Perform wavelet decomposition\n",
    "    coeffs = pywt.wavedec(cir_values, wavelet, level=level)\n",
    "\n",
    "    # Set the detail coefficients to zero\n",
    "    for i in range(1, len(coeffs)):\n",
    "        coeffs[i] = pywt.threshold(coeffs[i], value=0.5, mode='soft')\n",
    "\n",
    "    # Perform wavelet reconstruction\n",
    "    denoised_cir = pywt.waverec(coeffs, wavelet)\n",
    "\n",
    "    return denoised_cir\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.466282Z",
     "start_time": "2024-03-20T11:36:22.462030Z"
    }
   },
   "id": "fe3089568e99a58d",
   "execution_count": 78
  },
  {
   "cell_type": "markdown",
   "source": [
    "## deconvolve_cir Function\n",
    "\n",
    "The `deconvolve_cir` function applies the Richardson-Lucy deconvolution algorithm to deconvolve the Channel Impulse Response (CIR) values. \n",
    "\n",
    "In the context of signal processing, deconvolution is the process of reversing the effects of convolution on a signal. Convolution is a mathematical operation that blends two functions together and is often used to describe the effect of a linear time-invariant system on a signal. Deconvolution, therefore, attempts to recover the original signal that was convolved with the system's impulse response to produce the current signal.\n",
    "\n",
    "The Richardson-Lucy algorithm is an iterative method for deconvolution. It is particularly suitable for cases where the impulse response of the system (also known as the Point Spread Function, or PSF) is known, and the noise is Poissonian (such as in astronomical images). \n",
    "\n",
    "The algorithm works by iteratively refining an estimate of the original signal. In each iteration, it performs a prediction step where it convolves the current estimate with the PSF to predict the observed signal, and a correction step where it computes the ratio of the observed signal to the predicted signal, convolves this ratio with the PSF, and multiplies the result with the current estimate to get the next estimate.\n",
    "\n",
    "This process is repeated for a fixed number of iterations, or until the estimate converges to a stable solution. The result is a denoised estimate of the original signal.\n",
    "\n",
    "The mathematical formulas involved in the Richardson-Lucy deconvolution algorithm are as follows:\n",
    "\n",
    "1. **Prediction Step:**\n",
    "\n",
    "    The current estimate of the latent image `x` is convolved with the PSF `h` to predict the observed image `y`. This can be represented as:\n",
    "\n",
    "    $$\n",
    "    y = h \\ast x\n",
    "    $$\n",
    "\n",
    "    where $\\ast$ denotes the convolution operation.\n",
    "\n",
    "2. **Correction Step:**\n",
    "\n",
    "The ratio of the observed image $y$ to the predicted image $y'$ is computed, then the PSF $h$ is convolved with this ratio and multiplied with the current estimate $x$ to get the next estimate $x'$. This can be represented as:\n",
    "\n",
    "$$\n",
    "x' = x \\cdot (h \\ast \\left(\\frac{y}{y'}\\right))\n",
    "$$\n",
    "\n",
    "where $\\div$ denotes element-wise division and $\\ast$ denotes the convolution operation.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "e1edd5ef4f54e752"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "\n",
    "def deconvolve_cir(cir_values, psf=None, iterations=50):\n",
    "    # If no point spread function is provided, create a simple one\n",
    "    if psf is None:\n",
    "        psf = np.ones((5,)) / 5\n",
    "\n",
    "    # Perform Richardson-Lucy deconvolution\n",
    "    deconvolved_cir = restoration.richardson_lucy(cir_values, psf, num_iter=iterations)\n",
    "\n",
    "    return deconvolved_cir"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.544990Z",
     "start_time": "2024-03-20T11:36:22.541525Z"
    }
   },
   "id": "670e8c2cf19126ea",
   "execution_count": 79
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "\n",
    "def clean_data(data):\n",
    "    print(\"Starting data cleaning process...\")\n",
    "    \n",
    "    # print(\"Before Cleaning\")\n",
    "    # stat_analysis_and_plots(data)\n",
    "\n",
    "    # Calculate total number of missing values in the data\n",
    "    total_missing = data.isnull().sum().sum()\n",
    "    print(f\"Total number of missing values: {total_missing}\")\n",
    "\n",
    "    # Data has no missing values\n",
    "    data = data.dropna()\n",
    "    print(\"Missing values dropped.\")\n",
    "\n",
    "    # Data has no duplicate rows\n",
    "    data = data.drop_duplicates()\n",
    "    print(\"Duplicate rows dropped.\")\n",
    "\n",
    "    # Convert 'NLOS' column to integer data type (0 for LOS, 1 for NLOS)\n",
    "    data['NLOS'] = data['NLOS'].astype(int)\n",
    "    print(\"'NLOS' column converted to integer data type.\")\n",
    "    \n",
    "    # Print line where CIR_PWR is 0\n",
    "    print(f\"Line where CIR_PWR is 0: {data[data['CIR_PWR'] == 0]}\")\n",
    "    \n",
    "    # Calculate the expression inside the log10 function\n",
    "    expression = (data['CIR_PWR'] * (2**17)) / (data['RXPACC']**2)\n",
    "\n",
    "    # If the expression is 0, set 'RX_Level' to 0\n",
    "    zero_indices = expression == 0\n",
    "    data.loc[zero_indices, 'RX_Level'] = 0\n",
    "\n",
    "    # For the rest of the data where the expression is not 0, calculate 'RX_Level'\n",
    "    # First, update the 'expression' and 'data' to exclude zero_indices\n",
    "    expression = expression.loc[~zero_indices]\n",
    "    data = data.loc[~zero_indices]\n",
    "\n",
    "    # Now, calculate 'RX_Level' for the rest of the data\n",
    "    data['RX_Level'] = 10 * np.log10(expression) - data['PRFR']\n",
    "\n",
    "    # Calculate the median of 'RX_Level'\n",
    "    median = data['RX_Level'].median()\n",
    "\n",
    "    # Create the boolean mask on the same DataFrame 'data'\n",
    "    zero_indices = (data['RX_Level'] == 0)\n",
    "\n",
    "    # Replace zero values in 'RX_Level' with the median\n",
    "    data.loc[zero_indices, 'RX_Level'] = median\n",
    "\n",
    "    print(\"New feature 'RX_Level' created.\")\n",
    "\n",
    "    # Calculate new feature 'First_Path_Power_Level'\n",
    "    data['First_Path_Power_Level'] = (10 * np.log10(\n",
    "        (data['FP_AMP1'] ** 2 + data['FP_AMP2'] ** 2 + data['FP_AMP3'] ** 2) / (data['RXPACC'] ** 2))) - 64\n",
    "    print(\"New feature 'First_Path_Power_Level' calculated.\")\n",
    "    data.drop(['FP_AMP1', 'FP_AMP2', 'FP_AMP3', 'RXPACC', 'PRFR'], axis=1, inplace=True)\n",
    "\n",
    "    # Calculate SNR as the ratio of 'CIR_PWR' to 'STDEV_NOISE' for each data point\n",
    "    data['SNR'] = data['CIR_PWR'] / data['STDEV_NOISE']\n",
    "    print(\"New feature 'SNR' created.\")\n",
    "    data.drop(['CIR_PWR', 'STDEV_NOISE'], axis=1, inplace=True)\n",
    "    \n",
    "    plot_histogram(data, 'First_Path_Power_Level')\n",
    "    plot_histogram(data, 'RX_Level')\n",
    "\n",
    "    # One-hot encode categorical features\n",
    "    categorical_features = ['CH', 'FRAME_LEN', 'PREAM_LEN', 'BITRATE']\n",
    "    encoder = LabelEncoder()\n",
    "    for feature in categorical_features:\n",
    "        data[feature] = encoder.fit_transform(data[feature])\n",
    "    print(\"Categorical features one-hot encoded.\")\n",
    "\n",
    "    # Extract the 'CIR' columns\n",
    "    cir_columns = [col for col in data.columns if 'CIR' in col]\n",
    "    cir_data = data[cir_columns]    \n",
    "    print(\"'CIR' columns extracted.\")\n",
    "    \n",
    "    # Convert 'CIR' columns to float\n",
    "    cir_data = cir_data.astype(float)\n",
    "    print(\"'CIR' columns converted to float.\")\n",
    "    \n",
    "    # Denoise 'CIR' columns\n",
    "    denoised_cir_data = cir_data.apply(denoise_cir)\n",
    "    # denoised_cir_data = cir_data.apply(deconvolve_cir)\n",
    "    print(\"'CIR' columns denoised.\")\n",
    "    \n",
    "    # Replace original 'CIR' columns with denoised data\n",
    "    data[cir_columns] = denoised_cir_data\n",
    "    print(\"Original 'CIR' columns replaced with denoised data.\")\n",
    "    \n",
    "    # List of columns to check for unique values\n",
    "    columns_to_check = ['CH', 'PREAM_LEN', 'BITRATE']\n",
    "\n",
    "    # Iterate over the columns\n",
    "    for column in columns_to_check:\n",
    "        # If the column has only one unique value, drop it\n",
    "        if data[column].nunique() == 1:\n",
    "            data = data.drop(column, axis=1)\n",
    "            print(f\"Column '{column}' dropped due to having only one unique value.\")\n",
    "\n",
    "    # Print the shape of the cleaned data\n",
    "    print(f\"Cleaned data shape: {data.shape}\")\n",
    "\n",
    "    # print(\"After Cleaning\")\n",
    "    # stat_analysis_and_plots(data)\n",
    "    \n",
    "    print(\"Data cleaning process completed.\")\n",
    "    \n",
    "    # Return the cleaned data\n",
    "    return data"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:22.653819Z",
     "start_time": "2024-03-20T11:36:22.643573Z"
    }
   },
   "id": "685463c2d6065b08",
   "execution_count": 80
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data from pickle file...\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 1000x500 with 1 Axes>",
      "image/png": "[encoded data, 67132 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 6000x3000 with 1 Axes>",
      "image/png": "[encoded data, 285556 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 6000x3000 with 1 Axes>",
      "image/png": "[encoded data, 289944 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data loaded successfully.\n",
      "First few rows of the data:\n",
      "   NLOS  RANGE  FP_IDX  MAX_NOISE  FRAME_LEN  PREAM_LEN        CIR0  \\\n",
      "0     1   6.18   749.0     3668.0          2          0  747.646447   \n",
      "1     1   4.54   741.0     1031.0          2          0  123.353553   \n",
      "2     1   4.39   744.0      796.0          0          0  439.646447   \n",
      "3     1   1.27   748.0     1529.0          2          0  233.353553   \n",
      "4     0   1.16   743.0     2022.0          0          0   82.353553   \n",
      "\n",
      "         CIR1        CIR2        CIR3  ...     CIR1010     CIR1011  \\\n",
      "0  367.353553  744.646447  717.646447  ...  726.646447  367.353553   \n",
      "1  470.646447  409.353553  332.353553  ...  491.353553  403.646447   \n",
      "2  447.646447  130.353553   96.353553  ...  271.646447   73.353553   \n",
      "3  239.353553  175.646447  532.646447  ...  225.353553  154.646447   \n",
      "4  219.646447  110.646447  293.353553  ...  132.353553  131.353553   \n",
      "\n",
      "      CIR1012     CIR1013     CIR1014     CIR1015   RX_Level  \\\n",
      "0  802.646447  818.646447  466.646447  767.646447 -33.465374   \n",
      "1  334.353553  210.353553  102.353553    0.353553 -34.892422   \n",
      "2  125.353553  169.353553  182.353553    0.000000 -38.436975   \n",
      "3  171.646447  277.646447  317.646447    0.000000 -21.918230   \n",
      "4  102.353553  126.353553  162.646447    0.353553 -21.603535   \n",
      "\n",
      "   First_Path_Power_Level         SNR  Total_Distance  \n",
      "0              -39.074609   37.700000   234035.764530  \n",
      "1              -53.179491   95.794118   127050.760471  \n",
      "2              -49.812107   55.384615   115175.236492  \n",
      "3              -25.863535  200.859375   101750.988607  \n",
      "4              -23.232193  152.723684   119190.284232  \n",
      "\n",
      "[5 rows x 1026 columns]\n",
      "Column headers:\n",
      "Index(['NLOS', 'RANGE', 'FP_IDX', 'MAX_NOISE', 'FRAME_LEN', 'PREAM_LEN',\n",
      "       'CIR0', 'CIR1', 'CIR2', 'CIR3',\n",
      "       ...\n",
      "       'CIR1010', 'CIR1011', 'CIR1012', 'CIR1013', 'CIR1014', 'CIR1015',\n",
      "       'RX_Level', 'First_Path_Power_Level', 'SNR', 'Total_Distance'],\n",
      "      dtype='object', length=1026)\n"
     ]
    }
   ],
   "source": [
    "import pickle\n",
    "\n",
    "# File='data_original.pkl'\n",
    "File = 'data.pkl'\n",
    "\n",
    "# Check if the file exists\n",
    "if os.path.exists(File):\n",
    "    # If the file exists, load it\n",
    "    print(\"Loading data from pickle file...\")\n",
    "    with open(File, 'rb') as f:\n",
    "        data = pickle.load(f)\n",
    "        # plot_features(data, data['NLOS'], \"First_Path_Power_Level\", \"RX_Level\")\n",
    "        # plot_features(data, data['NLOS'], \"SNR\", \"RX_Level\")\n",
    "        # plot_features(data, data['NLOS'], \"SNR\", \"First_Path_Power_Level\")\n",
    "        snr_graph(data)\n",
    "        cir_graphs(data)\n",
    "    print(\"Data loaded successfully.\")\n",
    "else:\n",
    "    # If the file doesn't exist, load and clean the data\n",
    "    print(\"Pickle file not found. Loading and cleaning data...\")\n",
    "    data = load_data(DATASET_DIR)\n",
    "    cir_graphs(data)\n",
    "    data = clean_data(data)\n",
    "    plot_features(data, data['NLOS'], \"First_Path_Power_Level\", \"RX_Level\")\n",
    "    snr_graph(data)\n",
    "    cir_graphs(data)\n",
    "    print(calculate_total_distance(data))\n",
    "    print(\"Data loaded and cleaned successfully.\")\n",
    "    print(\"Saving cleaned data to pickle file...\")\n",
    "    with open(File, 'wb') as f:\n",
    "        pickle.dump(data, f)\n",
    "    print(\"Cleaned data saved to pickle file successfully.\")\n",
    "\n",
    "print(\"First few rows of the data:\")\n",
    "print(data.head())\n",
    "\n",
    "# Print Headers\n",
    "print(\"Column headers:\")\n",
    "print(data.columns)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.311103Z",
     "start_time": "2024-03-20T11:36:22.707131Z"
    }
   },
   "id": "79c2c23691b26753",
   "execution_count": 81
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "MODEL_DIR = './models'\n",
    "\n",
    "\n",
    "def train_and_save_model(classifier, X_train, y_train, file_name):\n",
    "    if not os.path.exists(MODEL_DIR):\n",
    "        os.makedirs(MODEL_DIR)\n",
    "\n",
    "    file_path = os.path.join(MODEL_DIR, file_name)\n",
    "\n",
    "    # Check if the file exists\n",
    "    if not os.path.exists(file_path):\n",
    "        print(f\"Training the model and saving it to {file_path}\")\n",
    "        # Train the classifier\n",
    "        classifier.fit(X_train, y_train)\n",
    "\n",
    "        # Save the trained model as a pickle string.\n",
    "        saved_model = pickle.dumps(classifier)\n",
    "\n",
    "        # Save the pickled model to a file\n",
    "        with open(file_path, 'wb') as file:\n",
    "            file.write(saved_model)\n",
    "\n",
    "    # Load the pickled model from the file\n",
    "    with open(file_path, 'rb') as file:\n",
    "        loaded_model = pickle.load(file)\n",
    "\n",
    "    return loaded_model"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.316699Z",
     "start_time": "2024-03-20T11:36:26.312417Z"
    }
   },
   "id": "12e16974341e6266",
   "execution_count": 82
  },
  {
   "cell_type": "markdown",
   "source": [
    "The selected code is performing data standardization, which is a common preprocessing step in many machine learning workflows. \n",
    "\n",
    "The purpose of standardization is to transform the data such that it has a mean of 0 and a standard deviation of 1. This is done to ensure that all features have the same scale, which is a requirement for many machine learning algorithms.\n",
    "\n",
    "The mathematical formulas used in this process are as follows:\n",
    "\n",
    "1. Calculate the mean (μ) of the data:\n",
    "\n",
    "$$\n",
    "\\mu = \\frac{1}{n} \\sum_{i=1}^{n} x_i\n",
    "$$\n",
    "Where:\n",
    "- $n$ is the number of observations in the data\n",
    "- $x_i$ is the value of the $i$-th observation\n",
    "- $\\sum$ denotes the summation over all observations\n",
    "\n",
    "2. Standardize the data by subtracting the mean from each observation and dividing by the standard deviation:\n",
    "\n",
    "$$\n",
    "\\text{Data}_i = \\frac{x_i - \\mu}{\\sigma}\n",
    "$$\n",
    "Where:\n",
    "- $\\text{Data}_i$ is the standardized value of the $i$-th observation\n",
    "- $\\sigma$ is the standard deviation of the data\n",
    "- $x_i$ is the value of the $i$-th observation\n",
    "- $\\mu$ is the mean of the data\n",
    "\n",
    "The `StandardScaler` class from the `sklearn.preprocessing` module is used to perform this standardization. The `fit_transform` method is used to calculate the mean and standard deviation of the data and then perform the standardization.\n",
    "\n",
    "**Note:** By setting the explained variance to 0.95, we are saying that we want to choose the smallest number of principal components such that 95% of the variance in the original data is retained. This means that the transformed data will retain 95% of the information of the original data, while potentially having fewer dimensions.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "b36814c942066d6"
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Data Mining / Machine Learning\n",
    "\n",
    "### I. Supervised Learning\n",
    "- **Decision**: Supervised learning is used due to the labeled dataset.\n",
    "- **Algorithm**: Random Forest Classifier is preferred for its performance in classification tasks.\n",
    "\n",
    "### II. Training/Test Split Ratio\n",
    "- **Decision**: 70:30 split is chosen for training/test dataset.\n",
    "- **Reasoning**: This split ensures sufficient data for training and testing.\n",
    "\n",
    "### III. Performance Metrics\n",
    "- **Classification Accuracy**: Measures the proportion of correctly classified instances.\n",
    "- **Confusion Matrix**: Provides a summary of predicted and actual classes.\n",
    "- **Classification Report**: Provides detailed metrics such as precision, recall, F1-score, and support for each class.\n",
    "\n",
    "The Random Forest Classifier is trained on the training set and evaluated on the test set using accuracy and classification report metrics.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "8fefd253728ea2f0"
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Split the data into training and testing sets\n",
    "\n",
    "The next step is to split the data into training and testing sets. This is a common practice in machine learning, where the training set is used to train the model, and the testing set is used to evaluate its performance.\n",
    "\n",
    "We will use the `train_test_split` function from the `sklearn.model_selection` module to split the data into training and testing sets. We will use 70% of the data for training and 30% for testing, which is a common split ratio."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "7d64d6490fa1c2c2"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.model_selection import train_test_split\n",
    "# from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "# Assuming 'NLOS' is your target column\n",
    "# y = data['NLOS']\n",
    "\n",
    "# Convert labels to categorical one-hot encoding\n",
    "# y_categorical = to_categorical(y, num_classes=2)\n",
    "\n",
    "# Now split the data\n",
    "# X_train, X_test, y_train, y_test = train_test_split(data, y_categorical, test_size=0.2)\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.322428Z",
     "start_time": "2024-03-20T11:36:26.318009Z"
    }
   },
   "id": "54d2a6506b584a03",
   "execution_count": 83
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Train a Random Forest Classifier\n",
    "\n",
    "The next step is to train a machine learning model on the training data. We will use the `RandomForestClassifier` class from the `sklearn.ensemble` module to train a random forest classifier.\n",
    "\n",
    "The random forest classifier is an ensemble learning method that operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees.\n",
    "\n",
    "We will use the `fit` method of the `RandomForestClassifier` object to train the model on the training data."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "ab55160e30fd6f99"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.ensemble import RandomForestClassifier\n",
    "# \n",
    "# # Initialize the classifier with parameters to prevent overfitting\n",
    "# classifier = RandomForestClassifier(n_estimators=200, max_depth=10, min_samples_split=10, min_samples_leaf=5, max_features='sqrt')\n",
    "# \n",
    "# loaded_model = train_and_save_model(classifier, X_train, y_train, 'random_forest_classifier.pkl')\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.327125Z",
     "start_time": "2024-03-20T11:36:26.324423Z"
    }
   },
   "id": "dc485f3de9f8936f",
   "execution_count": 84
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Evaluate the Model\n",
    "\n",
    "To evaluate the performance of the trained model on the testing data, we will use the `predict` method of the `RandomForestClassifier` object to make predictions on the testing data. We will then use the `accuracy_score` and `classification_report` functions from the `sklearn.metrics` module to calculate the accuracy and generate a classification report.\n",
    "\n",
    "- **Accuracy:** The accuracy score function calculates the proportion of correctly classified instances.\n",
    "\n",
    "- **Precision:** The ratio of correctly predicted positive observations to the total predicted positive observations. It is calculated as:\n",
    "\n",
    "  $$\n",
    "  \\text{Precision} = \\frac{\\text{True Positives}}{\\text{True Positives} + \\text{False Positives}}\n",
    "  $$\n",
    "\n",
    "- **Recall:** The ratio of correctly predicted positive observations to all observations in the actual class. It is calculated as:\n",
    "\n",
    "  $$\n",
    "  \\text{Recall} = \\frac{\\text{True Positives}}{\\text{True Positives} + \\text{False Negatives}}\n",
    "  $$\n",
    "\n",
    "- **F1 Score:** The weighted average of precision and recall. It is calculated as:\n",
    "\n",
    "  $$\n",
    "  \\text{F1 Score} = 2 \\times \\frac{\\text{Precision} \\times \\text{Recall}}{\\text{Precision} + \\text{Recall}}\n",
    "  $$\n",
    "\n",
    "- **Support:** The number of actual occurrences of the class in the dataset.\n",
    "\n",
    "The classification report provides a summary of the precision, recall, F1-score, and support for each class in the testing data, giving insight into how well the model is performing for each class.\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "424cc5954c9e81cc"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "\n",
    "# Make predictions on the test set using the loaded model\n",
    "# y_pred = loaded_model.predict(X_test)\n",
    "# \n",
    "# # Evaluate the loaded model\n",
    "# accuracy = accuracy_score(y_test, y_pred)\n",
    "# classification_rep = classification_report(y_test, y_pred)\n",
    "# cross_val_score = cross_val_score(loaded_model, X_test, y_test, cv=5)\n",
    "# \n",
    "# print(f\"Accuracy: {accuracy}\")\n",
    "# print(f\"Classification Report:\\n{classification_rep}\")\n",
    "# print(f\"Cross Validation Score: {cross_val_score}\")\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.331098Z",
     "start_time": "2024-03-20T11:36:26.328328Z"
    }
   },
   "id": "702b4f40dda16736",
   "execution_count": 85
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Visualize a Decision Tree from the Random Forest\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "41957f9babb74a3"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.tree import plot_tree\n",
    "# import matplotlib.pyplot as plt\n",
    "# \n",
    "# # Select one tree from the forest\n",
    "# estimator = loaded_model.estimators_[0]\n",
    "# \n",
    "# plt.figure(figsize=(100, 100))\n",
    "# plot_tree(estimator,\n",
    "#           filled=True,\n",
    "#           rounded=True,\n",
    "#           class_names=['NLOS', 'LOS'],\n",
    "#           feature_names=data.columns,\n",
    "#           max_depth=5)  # Limit the depth of the tree\n",
    "# plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.334924Z",
     "start_time": "2024-03-20T11:36:26.332269Z"
    }
   },
   "id": "1f6f826d6234591c",
   "execution_count": 86
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Support Vector Machine (SVM)"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "eef3be2c3026a909"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# import os\n",
    "# from sklearn.svm import SVC\n",
    "# import pickle\n",
    "# \n",
    "# svm = SVC(kernel='linear', random_state=42, verbose=True)\n",
    "# loaded_model = train_and_save_model(svm, X_train, y_train, 'svm_classifier.pkl')\n",
    "# \n",
    "# # Predict the labels for the test set with each model\n",
    "# y_pred_svm = loaded_model.predict(X_test)\n",
    "# \n",
    "# # Calculate the accuracy of each model\n",
    "# accuracy_svm = accuracy_score(y_test, y_pred_svm)\n",
    "# \n",
    "# # Print the accuracy of each model\n",
    "# print(f\"Accuracy of SVM: {accuracy_svm}\")"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.338799Z",
     "start_time": "2024-03-20T11:36:26.336101Z"
    }
   },
   "id": "c970b0c1593d955c",
   "execution_count": 87
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Logistic Regression"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "cccaf1db0d5060a8"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.linear_model import LogisticRegression\n",
    "# from sklearn.model_selection import cross_val_score\n",
    "# \n",
    "# # Logistic Regression with L2 regularization\n",
    "# log_reg = LogisticRegression(penalty='l2', C=0.1)\n",
    "# \n",
    "# # Use the train_and_save_model function to train and save the model\n",
    "# loaded_model = train_and_save_model(log_reg, X_train, y_train, 'logistic_regression_model.pkl')"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.342498Z",
     "start_time": "2024-03-20T11:36:26.339857Z"
    }
   },
   "id": "ee7506f4aa805faf",
   "execution_count": 88
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "\n",
    "# # Predict on the test set\n",
    "# y_pred_log_reg = loaded_model.predict(X_test)\n",
    "# \n",
    "# # Calculate accuracy\n",
    "# accuracy_log_reg = accuracy_score(y_test, y_pred_log_reg)\n",
    "# print(f\"Accuracy of Logistic Regression: {accuracy_log_reg}\")\n",
    "# \n",
    "# # Perform 5-fold cross validation\n",
    "# scores = cross_val_score(log_reg, X_train, y_train, cv=5)\n",
    "# print(f\"Cross-validated scores: {scores}\")"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.346181Z",
     "start_time": "2024-03-20T11:36:26.343627Z"
    }
   },
   "id": "a44d38efa4b86d93",
   "execution_count": 89
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.metrics import roc_curve, auc\n",
    "# import matplotlib.pyplot as plt\n",
    "# \n",
    "# # Compute ROC curve and ROC area for each class\n",
    "# fpr, tpr, _ = roc_curve(y_test, y_pred_log_reg)\n",
    "# roc_auc = auc(fpr, tpr)\n",
    "# \n",
    "# plt.figure()\n",
    "# lw = 2\n",
    "# plt.plot(fpr, tpr, color='darkorange',\n",
    "#          lw=lw, label='ROC curve (area = %0.2f)' % roc_auc)\n",
    "# plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "# plt.xlim([0.0, 1.0])\n",
    "# plt.ylim([0.0, 1.05])\n",
    "# plt.xlabel('False Positive Rate')\n",
    "# plt.ylabel('True Positive Rate')\n",
    "# plt.title('Receiver Operating Characteristic')\n",
    "# plt.legend(loc=\"lower right\")\n",
    "# plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.352459Z",
     "start_time": "2024-03-20T11:36:26.349609Z"
    }
   },
   "id": "a3646a4965b0707c",
   "execution_count": 90
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Gradient Boosting Classifier"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "aeaf5eeffa7ec104"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.ensemble import GradientBoostingClassifier\n",
    "# \n",
    "# # Gradient Boosting Classifier\n",
    "# gbc = GradientBoostingClassifier()\n",
    "# \n",
    "# # Use the train_and_save_model function to train and save the model\n",
    "# loaded_model = train_and_save_model(gbc, X_train, y_train, 'gradient_boosting_classifier.pkl')\n",
    "# "
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.355930Z",
     "start_time": "2024-03-20T11:36:26.353605Z"
    }
   },
   "id": "c7ecae5d021ad44f",
   "execution_count": 91
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# y_pred_gbc = loaded_model.predict(X_test)\n",
    "# accuracy_gbc = accuracy_score(y_test, y_pred_gbc)\n",
    "# print(f\"Accuracy of Gradient Boosting Classifier: {accuracy_gbc}\")\n",
    "# "
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.359480Z",
     "start_time": "2024-03-20T11:36:26.357096Z"
    }
   },
   "id": "4a8a1c3a7289ef7a",
   "execution_count": 92
  },
  {
   "cell_type": "markdown",
   "source": [
    "# K-Nearest Neighbors (KNN, K=15)\n",
    "\n",
    "This code block is implementing the K-Nearest Neighbors (KNN) algorithm for classification. The KNN algorithm is a type of instance-based learning, or lazy learning, where the function is only approximated locally and all computation is deferred until function evaluation.  \n",
    "\n",
    "The KNN algorithm works by finding the distances between a query and all the examples in the data, selecting the specified number examples (K) closest to the query, then votes for the most frequent label (in the case of classification) or averages the labels (in the case of regression).  \n",
    "\n",
    "The number of neighbors, K, is set to 15 in this case. This means that the algorithm looks at the 15 nearest neighbors to decide the class of the test instance.  \n",
    "\n",
    "The mathematical concept behind KNN is the Euclidean distance. Given two points P1(x1, y1) and P2(x2, y2) in a 2D space, the Euclidean distance between P1 and P2 is calculated as:\n",
    "\n",
    "$$\n",
    "\\text{Distance} = \\sqrt{(x2 - x1)^2 + (y2 - y1)^2}\n",
    "$$\n",
    "In higher dimensional space, the formula is generalized as:\n",
    "$$\n",
    "\\text{Distance} = \\sqrt{\\sum_{i=1}^{n} (x_i - y_i)^2}\n",
    "$$\n",
    "Where:\n",
    "- $n$ is the number of dimensions\n",
    "- $x_i$ and $y_i$ are the $i$-th dimensions of the two points\n"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "25102568a6e5c457"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.neighbors import KNeighborsClassifier\n",
    "# \n",
    "# # K-Nearest Neighbors\n",
    "# knn = KNeighborsClassifier(n_neighbors=13)\n",
    "# loaded_model = train_and_save_model(knn, X_train, y_train, 'knn_classifier.pkl')\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.362915Z",
     "start_time": "2024-03-20T11:36:26.360642Z"
    }
   },
   "id": "705c62e64bf6d614",
   "execution_count": 93
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# y_pred_knn = loaded_model.predict(X_test)\n",
    "# accuracy_knn = accuracy_score(y_test, y_pred_knn)\n",
    "# print(f\"Accuracy of K-Nearest Neighbors: {accuracy_knn}\")"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.366441Z",
     "start_time": "2024-03-20T11:36:26.364032Z"
    }
   },
   "id": "cf4df4ef7bbfd74",
   "execution_count": 94
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.model_selection import GridSearchCV\n",
    "# \n",
    "# # Define the parameter values that should be searched\n",
    "# k_range = list(range(1, 31))\n",
    "# \n",
    "# # Create a parameter grid: map the parameter names to the values that should be searched\n",
    "# param_grid = dict(n_neighbors=k_range)\n",
    "# \n",
    "# # Instantiate the grid\n",
    "# grid = GridSearchCV(knn, param_grid, cv=10, scoring='accuracy')\n",
    "# \n",
    "# # Fit the grid with data\n",
    "# grid.fit(X_train, y_train)\n",
    "# \n",
    "# # View the complete results\n",
    "# grid.cv_results_\n",
    "# \n",
    "# # Examine the best model\n",
    "# print(grid.best_score_)\n",
    "# print(grid.best_params_)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.370553Z",
     "start_time": "2024-03-20T11:36:26.367583Z"
    }
   },
   "id": "faabcf63e34005a9",
   "execution_count": 95
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# import matplotlib.pyplot as plt\n",
    "# import numpy as np\n",
    "# \n",
    "# # Apply PCA to reduce dimensionality to 2D\n",
    "# pca = PCA(n_components=2)\n",
    "# X_test_2d = pca.fit_transform(X_test)\n",
    "# \n",
    "# # Print the number of features\n",
    "# print(f\"Original number of features: {X_test.shape[1]}, reduced number of features: {X_test_2d.shape[1]}\")\n",
    "# \n",
    "# # Create a scatter plot\n",
    "# plt.figure(figsize=(10, 7))\n",
    "# \n",
    "# # Create a color map\n",
    "# cmap = plt.cm.viridis\n",
    "# \n",
    "# # Plot NLOS points\n",
    "# nlos = plt.scatter(X_test_2d[y_pred_knn == 1, 0], X_test_2d[y_pred_knn == 1, 1], c='blue', label='NLOS')\n",
    "# \n",
    "# # Plot LOS points\n",
    "# los = plt.scatter(X_test_2d[y_pred_knn == 0, 0], X_test_2d[y_pred_knn == 0, 1], c='red', label='LOS')\n",
    "# \n",
    "# # Add labels\n",
    "# plt.xlabel('Principal Component 1')\n",
    "# plt.ylabel('Principal Component 2')\n",
    "# plt.title('2D Scatter Plot for LOS and NLOS')\n",
    "# \n",
    "# # Add a legend\n",
    "# plt.legend(handles=[nlos, los])\n",
    "# \n",
    "# plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.374844Z",
     "start_time": "2024-03-20T11:36:26.371738Z"
    }
   },
   "id": "2ed22b3fc59f74e6",
   "execution_count": 96
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.neighbors import KNeighborsClassifier\n",
    "# from sklearn.metrics import accuracy_score\n",
    "# import matplotlib.pyplot as plt\n",
    "# \n",
    "# # Define the list of numbers of neighbors (from 1-20)\n",
    "# num_neighbors = np.arange(1, 100, 2)\n",
    "# \n",
    "# # Initialize the lists to store the accuracies\n",
    "# train_acc = []\n",
    "# test_acc = []\n",
    "# \n",
    "# # Loop over the different numbers of neighbors\n",
    "# for k in num_neighbors:\n",
    "#     # Initialize the KNN classifier\n",
    "#     clf = KNeighborsClassifier(n_neighbors=k)\n",
    "#     \n",
    "#     # Fit the classifier on the training data\n",
    "#     clf.fit(X_train, y_train)\n",
    "#     \n",
    "#     # Make predictions on the training and test data\n",
    "#     y_pred_train = clf.predict(X_train)\n",
    "#     y_pred_test = clf.predict(X_test)\n",
    "#     \n",
    "#     # Calculate the accuracies\n",
    "#     train_acc.append(accuracy_score(y_train, y_pred_train))\n",
    "#     test_acc.append(accuracy_score(y_test, y_pred_test))\n",
    "# \n",
    "# # Plot the accuracies\n",
    "# plt.figure(figsize=(10, 5))\n",
    "# plt.plot(num_neighbors, train_acc, 'ro-', num_neighbors, test_acc, 'bv--')\n",
    "# plt.legend(['Training Accuracy', 'Test Accuracy'])\n",
    "# plt.xlabel('Number of Neighbors')\n",
    "# plt.ylabel('Accuracy')\n",
    "# plt.title('Training and Test Accuracy for Different Numbers of Neighbors in KNN')\n",
    "# plt.grid()\n",
    "# plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.379861Z",
     "start_time": "2024-03-20T11:36:26.375928Z"
    }
   },
   "id": "4ac86c268055c1b8",
   "execution_count": 97
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Naive Bayes"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "5b9b66f92968957c"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.naive_bayes import GaussianNB\n",
    "# \n",
    "# # Naive Bayes\n",
    "# nb = GaussianNB()\n",
    "# loaded_model = train_and_save_model(nb, X_train, y_train, 'naive_bayes_classifier.pkl')"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.383672Z",
     "start_time": "2024-03-20T11:36:26.381305Z"
    }
   },
   "id": "3d984228fb1d3026",
   "execution_count": 98
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# y_pred_nb = loaded_model.predict(X_test)\n",
    "# accuracy_nb = accuracy_score(y_test, y_pred_nb)\n",
    "# print(f\"Accuracy of Naive Bayes: {accuracy_nb}\")"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.387312Z",
     "start_time": "2024-03-20T11:36:26.384890Z"
    }
   },
   "id": "98cd350871bc3201",
   "execution_count": 99
  },
  {
   "cell_type": "markdown",
   "source": [
    "# K-Means Clustering"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "92c8498137a5e32e"
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.cluster import KMeans\n",
    "# \n",
    "# # K-Means Clustering\n",
    "# kmeans = KMeans(n_clusters=2, max_iter=600)\n",
    "# loaded_model = train_and_save_model(kmeans, X_train, y_train, 'kmeans_clustering.pkl')"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.390727Z",
     "start_time": "2024-03-20T11:36:26.388474Z"
    }
   },
   "id": "305a796294814705",
   "execution_count": 100
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# y_pred_kmeans = loaded_model.predict(X_test)\n",
    "# accuracy_kmeans = accuracy_score(y_test, y_pred_kmeans)\n",
    "# print(f\"Accuracy of K-Means Clustering: {accuracy_kmeans}\")\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.394465Z",
     "start_time": "2024-03-20T11:36:26.392061Z"
    }
   },
   "id": "494bb537046bf5a7",
   "execution_count": 101
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# labels = loaded_model.labels_\n",
    "# # Print the data table with the cluster labels\n",
    "# print(f\"Data table with cluster labels:\\n{pd.concat([X_test, pd.DataFrame({'Cluster': labels})], axis=1)}\")\n",
    "# \n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.398192Z",
     "start_time": "2024-03-20T11:36:26.395730Z"
    }
   },
   "id": "62401c8d1a4d61cc",
   "execution_count": 102
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.cluster import KMeans\n",
    "# from sklearn.metrics import accuracy_score\n",
    "# from sklearn.decomposition import PCA\n",
    "# from mpl_toolkits.mplot3d import Axes3D\n",
    "# import matplotlib.pyplot as plt\n",
    "# \n",
    "# # Define the range of cluster numbers\n",
    "# cluster_range = range(1, 15)\n",
    "# \n",
    "# # For each number of clusters\n",
    "# for n_clusters in cluster_range:\n",
    "#     # Create a KMeans model\n",
    "#     kmeans = KMeans(n_clusters=n_clusters, max_iter=600)\n",
    "# \n",
    "#     # Fit the model to the training data\n",
    "#     kmeans.fit(X_train)\n",
    "# \n",
    "#     # Make predictions on the test data\n",
    "#     y_pred_kmeans = kmeans.predict(X_test)\n",
    "# \n",
    "#     # Calculate the accuracy of the model\n",
    "#     accuracy_kmeans = accuracy_score(y_test, y_pred_kmeans)\n",
    "# \n",
    "#     # Print the number of clusters and the corresponding accuracy\n",
    "#     print(f\"Number of clusters: {n_clusters}, Accuracy: {accuracy_kmeans}\")\n",
    "# \n",
    "#     # Apply PCA to reduce dimensionality to 3D\n",
    "#     pca = PCA(n_components=3)\n",
    "#     X_test_3d = pca.fit_transform(X_test)\n",
    "# \n",
    "#     # Create a 3D scatter plot\n",
    "#     fig = plt.figure(figsize=(10, 7))\n",
    "#     ax = fig.add_subplot(111, projection='3d')\n",
    "# \n",
    "#     # Create a color map\n",
    "#     cmap = plt.cm.get_cmap('viridis', n_clusters)  # We use 'viridis' colormap and we specify that we have n_clusters\n",
    "# \n",
    "#     # Plot the points with colors according to their cluster assignment\n",
    "#     scatter = ax.scatter(X_test_3d[:, 0], X_test_3d[:, 1], X_test_3d[:, 2], c=y_pred_kmeans, cmap=cmap)\n",
    "# \n",
    "#     # Add labels\n",
    "#     ax.set_xlabel('Principal Component 1')\n",
    "#     ax.set_ylabel('Principal Component 2')\n",
    "#     ax.set_zlabel('Principal Component 3')\n",
    "#     plt.title(f'3D Visualization of {n_clusters} Clusters')\n",
    "# \n",
    "#     # Display the plot\n",
    "#     plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.403315Z",
     "start_time": "2024-03-20T11:36:26.399350Z"
    }
   },
   "id": "f0f5284581e70e6e",
   "execution_count": 103
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# from sklearn.decomposition import PCA\n",
    "# import matplotlib.pyplot as plt\n",
    "# \n",
    "# # Apply PCA to reduce dimensionality to 2D\n",
    "# pca = PCA(n_components=2)\n",
    "# X_test_2d = pca.fit_transform(X_test)\n",
    "# \n",
    "# # Predict the cluster labels for the data points you're plotting\n",
    "# labels = loaded_model.predict(X_test)\n",
    "# \n",
    "# # Create a scatter plot\n",
    "# plt.figure(figsize=(10, 7))\n",
    "# \n",
    "# # Create a color map\n",
    "# cmap = plt.cm.get_cmap('viridis', 2)  # We use 'viridis' colormap and we specify that we have 2 clusters\n",
    "# \n",
    "# # Plot the points with colors according to their cluster assignment\n",
    "# plt.scatter(X_test_2d[:, 0], X_test_2d[:, 1], c=labels, cmap=cmap)\n",
    "# \n",
    "# # Add labels\n",
    "# plt.xlabel('Principal Component 1')\n",
    "# plt.ylabel('Principal Component 2')\n",
    "# plt.title('2D Visualization of Clusters')\n",
    "# \n",
    "# # Display the plot\n",
    "# plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:36:26.407446Z",
     "start_time": "2024-03-20T11:36:26.404679Z"
    }
   },
   "id": "82c7ba8cbb2aa17a",
   "execution_count": 104
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Convolution Neural Network\n",
    "\n",
    "This code block is implementing a Convolutional Neural Network (CNN) for a classification task using TensorFlow.  The CNN is a class of deep learning neural networks, most commonly applied to analyzing visual imagery. They are also known as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics.  Here's a step-by-step breakdown of what the code does:  \n",
    "1. Data Preparation: The target column 'NLOS' is separated from the rest of the dataset. The target values are then encoded from categorical to numerical values using LabelEncoder. These numerical values are then one-hot encoded to create binary variables for each class.  \n",
    "2. Data Reshaping: The input data is reshaped to fit the model. Each data instance is reshaped to a 3D array where the third dimension represents the number of input channels, which is 1 in this case.  \n",
    "3. Data Splitting: The data is split into training and testing sets using a 80:20 ratio.  \n",
    "4. Model Creation: A Sequential model is created using Keras. This model is composed of the following layers:  \n",
    "5. Conv1D layers: These are convolutional layers that will convolve the input data with a set of learnable filters, each producing one feature map in the output. The kernel size is set to 3, and the activation function used is ReLU (Rectified Linear Unit).  \n",
    "6. MaxPooling1D layers: These layers are used to down-sample the input along its spatial dimensions (height and width). The pool size is set to 2.  \n",
    "7. Dense layers: These are fully connected layers. The first Dense layer has 64 units and uses the ReLU activation function. The second Dense layer has a number of units equal to the number of classes and uses the softmax activation function to output a probability distribution over the classes.  \n",
    "9. Model Compilation: The model is compiled with the Adam optimizer, categorical cross-entropy loss function, and accuracy as the evaluation metric.  \n",
    "10. Model Training: The model is trained on the training data for 10 epochs with a batch size of 32. The validation data is set to the testing set.  \n",
    "11. Model Evaluation: The model's performance is evaluated on the testing set and the accuracy is printed.  \n",
    "\n",
    "12. The mathematical concept behind the Convolutional layer (Conv1D) is the convolution operation, which is a mathematical operation on two functions that produces a third function. In the context of a CNN, the two functions are the input data and the kernel or filter. The convolution operation involves sliding the kernel across the input data and computing the dot product at each position.\n",
    "\n",
    "The mathematical formula for the convolution operation is:  $$ (f * g)(t) = \\int_{-\\infty}^{\\infty} f(\\tau)g(t - \\tau) d\\tau $$  Where:  \n",
    "$f$ and $g$ are the input data and kernel respectively\n",
    "$t$ is the position of the kernel\n",
    "$\\tau$ is a dummy integration variable\n",
    "In the context of a CNN, the integral is replaced by a sum over the discrete spatial dimensions (height and width) of the input data and kernel."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "862a9b7ee430a667"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30\n",
      "1050/1050 [==============================] - 26s 23ms/step - loss: 0.8091 - accuracy: 0.6635 - val_loss: 0.5219 - val_accuracy: 0.8042\n",
      "Epoch 2/30\n",
      "1050/1050 [==============================] - 26s 24ms/step - loss: 0.5925 - accuracy: 0.7790 - val_loss: 0.4711 - val_accuracy: 0.8326\n",
      "Epoch 3/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.5297 - accuracy: 0.8096 - val_loss: 0.4609 - val_accuracy: 0.8440\n",
      "Epoch 4/30\n",
      "1050/1050 [==============================] - 26s 25ms/step - loss: 0.5034 - accuracy: 0.8216 - val_loss: 0.4540 - val_accuracy: 0.8443\n",
      "Epoch 5/30\n",
      "1050/1050 [==============================] - 25s 24ms/step - loss: 0.4806 - accuracy: 0.8330 - val_loss: 0.4468 - val_accuracy: 0.8487\n",
      "Epoch 6/30\n",
      "1050/1050 [==============================] - 27s 25ms/step - loss: 0.4720 - accuracy: 0.8358 - val_loss: 0.4471 - val_accuracy: 0.8476\n",
      "Epoch 7/30\n",
      "1050/1050 [==============================] - 25s 24ms/step - loss: 0.4606 - accuracy: 0.8416 - val_loss: 0.4418 - val_accuracy: 0.8496\n",
      "Epoch 8/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.4517 - accuracy: 0.8451 - val_loss: 0.4389 - val_accuracy: 0.8519\n",
      "Epoch 9/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4418 - accuracy: 0.8500 - val_loss: 0.4360 - val_accuracy: 0.8527\n",
      "Epoch 10/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4380 - accuracy: 0.8526 - val_loss: 0.4345 - val_accuracy: 0.8535\n",
      "Epoch 11/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4379 - accuracy: 0.8535 - val_loss: 0.4312 - val_accuracy: 0.8540\n",
      "Epoch 12/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.4325 - accuracy: 0.8527 - val_loss: 0.4287 - val_accuracy: 0.8564\n",
      "Epoch 13/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4247 - accuracy: 0.8553 - val_loss: 0.4308 - val_accuracy: 0.8533\n",
      "Epoch 14/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4212 - accuracy: 0.8583 - val_loss: 0.4250 - val_accuracy: 0.8557\n",
      "Epoch 15/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4187 - accuracy: 0.8575 - val_loss: 0.4292 - val_accuracy: 0.8537\n",
      "Epoch 16/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4173 - accuracy: 0.8586 - val_loss: 0.4223 - val_accuracy: 0.8575\n",
      "Epoch 17/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.4112 - accuracy: 0.8608 - val_loss: 0.4208 - val_accuracy: 0.8555\n",
      "Epoch 18/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.4089 - accuracy: 0.8591 - val_loss: 0.4201 - val_accuracy: 0.8545\n",
      "Epoch 19/30\n",
      "1050/1050 [==============================] - 25s 24ms/step - loss: 0.4078 - accuracy: 0.8604 - val_loss: 0.4178 - val_accuracy: 0.8556\n",
      "Epoch 20/30\n",
      "1050/1050 [==============================] - 25s 24ms/step - loss: 0.4061 - accuracy: 0.8635 - val_loss: 0.4183 - val_accuracy: 0.8581\n",
      "Epoch 21/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.3997 - accuracy: 0.8644 - val_loss: 0.4131 - val_accuracy: 0.8568\n",
      "Epoch 22/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.3959 - accuracy: 0.8646 - val_loss: 0.4125 - val_accuracy: 0.8582\n",
      "Epoch 23/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.3980 - accuracy: 0.8621 - val_loss: 0.4088 - val_accuracy: 0.8598\n",
      "Epoch 24/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.3930 - accuracy: 0.8647 - val_loss: 0.4090 - val_accuracy: 0.8589\n",
      "Epoch 25/30\n",
      "1050/1050 [==============================] - 25s 24ms/step - loss: 0.3930 - accuracy: 0.8651 - val_loss: 0.4078 - val_accuracy: 0.8592\n",
      "Epoch 26/30\n",
      "1050/1050 [==============================] - 25s 24ms/step - loss: 0.3900 - accuracy: 0.8664 - val_loss: 0.4076 - val_accuracy: 0.8586\n",
      "Epoch 27/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.3877 - accuracy: 0.8665 - val_loss: 0.4098 - val_accuracy: 0.8582\n",
      "Epoch 28/30\n",
      "1050/1050 [==============================] - 24s 23ms/step - loss: 0.3849 - accuracy: 0.8691 - val_loss: 0.4071 - val_accuracy: 0.8607\n",
      "Epoch 29/30\n",
      "1050/1050 [==============================] - 23s 22ms/step - loss: 0.3857 - accuracy: 0.8682 - val_loss: 0.4058 - val_accuracy: 0.8601\n",
      "Epoch 30/30\n",
      "1050/1050 [==============================] - 24s 22ms/step - loss: 0.3871 - accuracy: 0.8673 - val_loss: 0.4040 - val_accuracy: 0.8630\n",
      "263/263 [==============================] - 1s 4ms/step\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Conv1D, MaxPooling1D, Flatten, Dense, Dropout, BatchNormalization\n",
    "from tensorflow.keras.callbacks import EarlyStopping\n",
    "from tensorflow.keras import regularizers\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from sklearn.metrics import classification_report\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Set random seed for reproducibility\n",
    "tf.random.set_seed(42)\n",
    "\n",
    "# Drop the target column 'NLOS' from the data and assign the remaining data to X\n",
    "X = data.drop('NLOS', axis=1)\n",
    "# Assign the target column 'NLOS' to y\n",
    "y = data['NLOS']\n",
    "\n",
    "# Split the data into training and testing sets with a 80:20 ratio\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "# Initialize a StandardScaler object\n",
    "scaler = StandardScaler()\n",
    "# Fit the scaler to the training data and transform it\n",
    "X_train = scaler.fit_transform(X_train)\n",
    "# Transform the testing data using the fitted scaler\n",
    "X_test = scaler.transform(X_test)\n",
    "\n",
    "# Initialize a Sequential model\n",
    "model = Sequential()\n",
    "\n",
    "# Add a Conv1D layer\n",
    "model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(X_train.shape[1], 1), kernel_regularizer=regularizers.l2(0.001)))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "# Add another Conv1D layer\n",
    "model.add(Conv1D(filters=32, kernel_size=3, activation='relu', kernel_regularizer=regularizers.l2(0.001)))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "# Add a Flatten layer\n",
    "model.add(Flatten())\n",
    "\n",
    "# Add a Dense layer\n",
    "model.add(Dense(16, activation='relu', kernel_regularizer=regularizers.l2(0.001)))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "# Add the output Dense layer\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "# Define early stopping\n",
    "early_stopping = EarlyStopping(monitor='val_loss', patience=10)\n",
    "\n",
    "# Compile the model\n",
    "model.compile(loss='binary_crossentropy', optimizer=Adam(learning_rate=0.0001), metrics=['accuracy'])\n",
    "\n",
    "# Train the model\n",
    "history = model.fit(X_train, y_train, epochs=30, batch_size=32, validation_data=(X_test, y_test), callbacks=[early_stopping])\n",
    "\n",
    "# Evaluate the model\n",
    "scores = model.evaluate(X_test, y_test, verbose=0)\n",
    "\n",
    "# Make predictions\n",
    "y_pred = model.predict(X_test)\n",
    "y_pred_classes = (y_pred > 0.5).astype(\"int32\")\n",
    "\n",
    "# Generate a classification report\n",
    "report = classification_report(y_test, y_pred_classes)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:48:39.730855Z",
     "start_time": "2024-03-20T11:36:26.408455Z"
    }
   },
   "id": "1c1dd203ad7db076",
   "execution_count": 105
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>",
      "image/png": "[encoded data, 42144 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 1200x600 with 1 Axes>",
      "image/png": "[encoded data, 50268 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.4039548933506012\n",
      "Test accuracy: 0.8629761934280396\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.91      0.87      4192\n",
      "           1       0.90      0.82      0.86      4208\n",
      "\n",
      "    accuracy                           0.86      8400\n",
      "   macro avg       0.87      0.86      0.86      8400\n",
      "weighted avg       0.87      0.86      0.86      8400\n"
     ]
    }
   ],
   "source": [
    "\n",
    "# Plot the training and validation accuracy over epochs\n",
    "plt.plot(history.history['accuracy'], 'ro-', history.history['val_accuracy'], 'bv--')\n",
    "plt.title('Training and Test Accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(['Training Accuracy', 'Test Accuracy'])\n",
    "plt.show()\n",
    "\n",
    "# Plot the training and validation loss over epochs\n",
    "plt.figure(figsize=(12, 6))\n",
    "plt.plot(history.history['loss'], label='Training Loss')\n",
    "plt.plot(history.history['val_loss'], label='Validation Loss')\n",
    "plt.title('Training and Validation Loss Over Time')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "# Print the testing loss and accuracy\n",
    "print('Test loss:', scores[0])\n",
    "print('Test accuracy:', scores[1])\n",
    "\n",
    "# Print the classification report\n",
    "print('Classification Report: \\n', report)\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:48:40.119743Z",
     "start_time": "2024-03-20T11:48:39.731977Z"
    }
   },
   "id": "89aa08d7d1866179",
   "execution_count": 106
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 1000x500 with 2 Axes>",
      "image/png": "[encoded data, 33628 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 1000x500 with 2 Axes>",
      "image/png": "[encoded data, 22224 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 500x500 with 2 Axes>",
      "image/png": "[encoded data, 25840 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>",
      "image/png": "[encoded data, 47124 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Conv1D, MaxPooling1D, Flatten, Dense, Dropout, BatchNormalization\n",
    "from tensorflow.keras.callbacks import EarlyStopping\n",
    "from tensorflow.keras import regularizers\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from sklearn.metrics import classification_report, confusion_matrix, roc_curve, auc\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# Visualize Weights and Biases\n",
    "for layer in model.layers:\n",
    "    if 'dense' in layer.name:\n",
    "        weights, biases = layer.get_weights()\n",
    "        plt.figure(figsize=(10, 5))\n",
    "        plt.subplot(1, 2, 1)\n",
    "        plt.hist(weights.flatten())\n",
    "        plt.title(f'{layer.name} weights')\n",
    "        plt.subplot(1, 2, 2)\n",
    "        plt.hist(biases.flatten())\n",
    "        plt.title(f'{layer.name} biases')\n",
    "        plt.show()\n",
    "\n",
    "\n",
    "# Confusion Matrix\n",
    "cm = confusion_matrix(y_test, y_pred_classes)\n",
    "plt.figure(figsize=(5, 5))\n",
    "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues')\n",
    "plt.title('Confusion matrix')\n",
    "plt.xlabel('Predicted class')\n",
    "plt.ylabel('True class')\n",
    "plt.show()\n",
    "\n",
    "# ROC Curve\n",
    "fpr, tpr, _ = roc_curve(y_test, y_pred)\n",
    "roc_auc = auc(fpr, tpr)\n",
    "plt.figure()\n",
    "lw = 2\n",
    "plt.plot(fpr, tpr, color='darkorange', lw=lw, label='ROC curve (area = %0.2f)' % roc_auc)\n",
    "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver Operating Characteristic')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show()\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:48:41.104783Z",
     "start_time": "2024-03-20T11:48:40.121419Z"
    }
   },
   "id": "dd49203934ca9cf6",
   "execution_count": 107
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 184508 characters]",
      "text/plain": "<IPython.core.display.Image object>"
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Plot the model\n",
    "from tensorflow.keras.utils import plot_model\n",
    "\n",
    "# Generate the plot\n",
    "plot_model(model, to_file='model_plot.png', show_shapes=True, show_layer_names=True)\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:48:41.231645Z",
     "start_time": "2024-03-20T11:48:41.109033Z"
    }
   },
   "id": "81f2d793ada5c410",
   "execution_count": 108
  },
  {
   "cell_type": "code",
   "outputs": [],
   "source": [
    "# Save the model\n",
    "model.save('CNN.keras')"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:48:41.298377Z",
     "start_time": "2024-03-20T11:48:41.233101Z"
    }
   },
   "id": "6b7329b28452b82a",
   "execution_count": 109
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Multi-Layer Perceptron (MLP)\n",
    "\n",
    "This code block is implementing a Multi-Layer Perceptron (MLP) for a binary classification task using TensorFlow. The MLP is a class of feedforward artificial neural network that consists of at least three layers of nodes: an input layer, a hidden layer, and an output layer. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called backpropagation for training.\n",
    "\n",
    "Here's a step-by-step breakdown of what the code does:\n",
    "\n",
    "1. **Data Preparation**: The target column 'NLOS' is separated from the rest of the dataset. The remaining data is assigned to X and the target column to y.\n",
    "\n",
    "2. **Data Splitting**: The data is split into training and testing sets using an 80:20 ratio.\n",
    "\n",
    "3. **Data Scaling**: A StandardScaler object is initialized and fitted to the training data. The training and testing data are then transformed using the fitted scaler.\n",
    "\n",
    "4. **Model Creation**: A Sequential model is created using Keras. This model is composed of the following layers:\n",
    "    - Dense layers: These are fully connected layers. The first Dense layer has 64 units and uses the ReLU activation function. The second and third Dense layers have 32 and 16 units respectively, and also use the ReLU activation function. The final Dense layer has 1 unit and uses the sigmoid activation function for binary classification.\n",
    "    - BatchNormalization layers: These layers are used to normalize the activations of the previous layer, which speeds up learning and provides some regularization, reducing generalization error.\n",
    "    - Dropout layers: These layers are used to prevent overfitting. They randomly set a fraction of input units to 0 at each update during training time.\n",
    "\n",
    "5. **Model Compilation**: The model is compiled with the Adam optimizer, binary cross-entropy loss function, and accuracy as the evaluation metric.\n",
    "\n",
    "6. **Model Training**: The model is trained on the training data for 20 epochs with a batch size of 32. The validation data is set to the testing set. Early stopping is used to stop training when the validation loss has not improved for 10 epochs.\n",
    "\n",
    "7. **Model Evaluation**: The model's performance is evaluated on the testing data and the loss and accuracy are printed. The model also makes predictions on the testing data, converts the predicted probabilities to binary outputs, and generates a classification report.\n",
    "\n",
    "8. **Visualization**: The training and validation accuracy and loss over epochs are plotted.\n",
    "\n",
    "The mathematical concept behind the Dense layer is the dot product operation, which is a mathematical operation that takes two equal-length sequences of numbers and returns a single number. In the context of a MLP, the two sequences are the input data and the weights of the neurons. The dot product operation involves multiplying each pair of input and weight and summing the result.\n",
    "\n",
    "The mathematical formula for the dot product operation is: $$ a \\cdot b = \\sum_{i=1}^{n} a_i b_i $$ Where:\n",
    "- $a$ and $b$ are the input data and weights respectively\n",
    "- $n$ is the number of dimensions (length of the sequences)\n",
    "- $a_i$ and $b_i$ are the $i$-th elements of the input data and weights respectively."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "42eff9445377f73c"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "1050/1050 [==============================] - 4s 3ms/step - loss: 1.0607 - accuracy: 0.5246 - val_loss: 0.8445 - val_accuracy: 0.6040\n",
      "Epoch 2/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.9423 - accuracy: 0.5647 - val_loss: 0.7906 - val_accuracy: 0.6873\n",
      "Epoch 3/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.8554 - accuracy: 0.6142 - val_loss: 0.7314 - val_accuracy: 0.7438\n",
      "Epoch 4/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.7918 - accuracy: 0.6648 - val_loss: 0.6835 - val_accuracy: 0.7739\n",
      "Epoch 5/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.7319 - accuracy: 0.7110 - val_loss: 0.6470 - val_accuracy: 0.7875\n",
      "Epoch 6/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.6907 - accuracy: 0.7468 - val_loss: 0.6125 - val_accuracy: 0.8017\n",
      "Epoch 7/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.6502 - accuracy: 0.7732 - val_loss: 0.5862 - val_accuracy: 0.8086\n",
      "Epoch 8/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.6240 - accuracy: 0.7870 - val_loss: 0.5660 - val_accuracy: 0.8151\n",
      "Epoch 9/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5995 - accuracy: 0.7994 - val_loss: 0.5473 - val_accuracy: 0.8213\n",
      "Epoch 10/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5735 - accuracy: 0.8137 - val_loss: 0.5342 - val_accuracy: 0.8252\n",
      "Epoch 11/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5655 - accuracy: 0.8177 - val_loss: 0.5221 - val_accuracy: 0.8264\n",
      "Epoch 12/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5485 - accuracy: 0.8227 - val_loss: 0.5098 - val_accuracy: 0.8298\n",
      "Epoch 13/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5358 - accuracy: 0.8277 - val_loss: 0.5021 - val_accuracy: 0.8342\n",
      "Epoch 14/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5218 - accuracy: 0.8307 - val_loss: 0.4914 - val_accuracy: 0.8355\n",
      "Epoch 15/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5076 - accuracy: 0.8371 - val_loss: 0.4864 - val_accuracy: 0.8352\n",
      "Epoch 16/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.5023 - accuracy: 0.8363 - val_loss: 0.4793 - val_accuracy: 0.8363\n",
      "Epoch 17/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.4901 - accuracy: 0.8388 - val_loss: 0.4747 - val_accuracy: 0.8361\n",
      "Epoch 18/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.4824 - accuracy: 0.8429 - val_loss: 0.4691 - val_accuracy: 0.8385\n",
      "Epoch 19/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.4719 - accuracy: 0.8455 - val_loss: 0.4645 - val_accuracy: 0.8365\n",
      "Epoch 20/20\n",
      "1050/1050 [==============================] - 2s 2ms/step - loss: 0.4657 - accuracy: 0.8467 - val_loss: 0.4613 - val_accuracy: 0.8371\n",
      "263/263 [==============================] - 0s 1ms/step\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 0 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 1200x600 with 1 Axes>",
      "image/png": "[encoded data, 66256 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 1200x600 with 1 Axes>",
      "image/png": "[encoded data, 66960 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 0 Axes>"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.4613083004951477\n",
      "Test accuracy: 0.8371428847312927\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.80      0.90      0.85      4192\n",
      "           1       0.89      0.77      0.83      4208\n",
      "\n",
      "    accuracy                           0.84      8400\n",
      "   macro avg       0.84      0.84      0.84      8400\n",
      "weighted avg       0.84      0.84      0.84      8400\n"
     ]
    }
   ],
   "source": [
    "# Import necessary libraries\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Dropout\n",
    "from tensorflow.keras.callbacks import EarlyStopping\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from tensorflow.keras.layers import BatchNormalization\n",
    "from tensorflow.keras import regularizers\n",
    "from tensorflow.keras.optimizers import Adam\n",
    "from sklearn.metrics import confusion_matrix, roc_curve, auc\n",
    "import seaborn as sns\n",
    "\n",
    "# Set random seed for reproducibility\n",
    "tf.random.set_seed(42)\n",
    "\n",
    "# Drop the target column 'NLOS' from the data and assign the remaining data to X\n",
    "X = data.drop('NLOS', axis=1)\n",
    "# Assign the target column 'NLOS' to y\n",
    "y = data['NLOS']\n",
    "\n",
    "# Split the data into training and testing sets with a 80:20 ratio\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "# Initialize a StandardScaler object\n",
    "scaler = StandardScaler()\n",
    "# Fit the scaler to the training data and transform it\n",
    "X_train = scaler.fit_transform(X_train)\n",
    "# Transform the testing data using the fitted scaler\n",
    "X_test = scaler.transform(X_test)\n",
    "\n",
    "# Initialize a Sequential model\n",
    "model = Sequential()\n",
    "# Add a Dense layer with 64 units, ReLU activation function and L2 regularization\n",
    "model.add(Dense(64, input_dim=X_train.shape[1], activation='relu', kernel_regularizer=regularizers.l2(0.001)))\n",
    "# Add a BatchNormalization layer to normalize the activations of the previous layer\n",
    "model.add(BatchNormalization())\n",
    "# Add a Dropout layer to prevent overfitting\n",
    "model.add(Dropout(0.5))\n",
    "# Add another Dense layer with 32 units, ReLU activation function and L2 regularization\n",
    "model.add(Dense(32, activation='relu', kernel_regularizer=regularizers.l2(0.001)))\n",
    "# Add another BatchNormalization layer\n",
    "model.add(BatchNormalization())\n",
    "# Add another Dropout layer\n",
    "model.add(Dropout(0.5))\n",
    "# Add another Dense layer with 16 units, ReLU activation function and L2 regularization\n",
    "model.add(Dense(16, activation='relu', kernel_regularizer=regularizers.l2(0.001)))\n",
    "# Add another BatchNormalization layer\n",
    "model.add(BatchNormalization())\n",
    "# Add another Dropout layer\n",
    "model.add(Dropout(0.5))\n",
    "# Add the output Dense layer with 1 unit and sigmoid activation function\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "# Define early stopping to stop training when the validation loss has not improved for 10 epochs\n",
    "early_stopping = EarlyStopping(monitor='val_loss', patience=10)\n",
    "\n",
    "# Compile the model with Adam optimizer, binary cross-entropy loss function and accuracy as the evaluation metric\n",
    "model.compile(loss='binary_crossentropy', optimizer=Adam(learning_rate=0.0001), metrics=['accuracy'])\n",
    "\n",
    "# Train the model on the training data for 20 epochs with a batch size of 32 and validate on the testing data\n",
    "history = model.fit(X_train, y_train, epochs=20, batch_size=32, validation_data=(X_test, y_test), callbacks=[early_stopping])\n",
    "\n",
    "# Evaluate the model on the testing data and store the loss and accuracy in 'scores'\n",
    "scores = model.evaluate(X_test, y_test, verbose=0)\n",
    "\n",
    "# Make predictions on the testing data\n",
    "y_pred = model.predict(X_test)\n",
    "# Convert the predicted probabilities to binary outputs\n",
    "y_pred_classes = (y_pred > 0.5).astype(\"int32\")\n",
    "# Generate a classification report\n",
    "report = classification_report(y_test, y_pred_classes)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12, 6))\n",
    "\n",
    "# Plot training & validation accuracy values\n",
    "ax.plot(history.history['accuracy'], 'ro-', label='Training Accuracy')\n",
    "ax.plot(history.history['val_accuracy'], 'bv--', label='Test Accuracy')\n",
    "\n",
    "# Plot training & validation loss values\n",
    "ax.plot(history.history['loss'], 'go-', label='Training Loss')\n",
    "ax.plot(history.history['val_loss'], 'yv--', label='Validation Loss')\n",
    "\n",
    "ax.set_title('Combined Graph')\n",
    "ax.set_ylabel('Values')\n",
    "ax.set_xlabel('Epoch')\n",
    "ax.legend()\n",
    "\n",
    "# Show the figure\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Create a figure for the accuracy plot\n",
    "fig1, ax1 = plt.subplots(figsize=(12, 6))\n",
    "\n",
    "# Plot training & validation accuracy values\n",
    "ax1.plot(history.history['accuracy'], 'ro-', label='Training Accuracy')\n",
    "ax1.plot(history.history['val_accuracy'], 'bv--', label='Test Accuracy')\n",
    "ax1.set_title('Model Accuracy')\n",
    "ax1.set_ylabel('Accuracy')\n",
    "ax1.set_xlabel('Epoch')\n",
    "ax1.legend()\n",
    "\n",
    "# Show the figure\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Create a figure for the loss plot\n",
    "fig2, ax2 = plt.subplots(figsize=(12, 6))\n",
    "\n",
    "# Plot training & validation loss values\n",
    "ax2.plot(history.history['loss'], 'ro-', label='Training Loss')\n",
    "ax2.plot(history.history['val_loss'], 'bv--', label='Validation Loss')\n",
    "ax2.set_title('Model Loss')\n",
    "ax2.set_ylabel('Loss')\n",
    "ax2.set_xlabel('Epoch')\n",
    "ax2.legend()\n",
    "\n",
    "# Show the figure\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Show the figure\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Print the testing loss and accuracy\n",
    "print('Test loss:', scores[0])\n",
    "print('Test accuracy:', scores[1])\n",
    "# Print the classification report\n",
    "print('Classification Report: \\n', report)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:49:34.073929Z",
     "start_time": "2024-03-20T11:48:41.299882Z"
    }
   },
   "id": "c8745832a585d5ec",
   "execution_count": 110
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Multi-Layer Perceptron (MLP) visualization\n",
    "This code block is used to visualize the performance of a trained Multi-Layer Perceptron (MLP) model. It generates three types of visualizations:\n",
    "\n",
    "1. Weights and Biases Visualization: This visualization is used to understand the distribution of weights and biases in the model's layers. For each layer in the model, if the layer is a dense layer, it retrieves the weights and biases, and plots histograms of their values. The x-axis of the histogram represents the value of the weights/biases and the y-axis represents the frequency of these values.\n",
    "\n",
    "2. Confusion Matrix: A confusion matrix is a table that is often used to describe the performance of a classification model on a set of test data for which the true values are known. It gives a more detailed breakdown of correct and incorrect classifications for each class.\n",
    "\n",
    "3. ROC Curve: The Receiver Operating Characteristic (ROC) curve is a plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. It is created by plotting the true positive rate (TPR) against the false positive rate (FPR).\n",
    "\n",
    "## Weights and Biases Evaluation\n",
    "\n",
    "The weights and biases of the MLP model layers were visualized to understand their distributions. The weights in all layers (dense, dense_1, dense_2, and dense_3) are not close to zero, indicating they are likely being updated during training and contributing to the model's learning. The weight distributions show a spread around zero, suggesting the model is capturing complex relationships in the data.\n",
    "\n",
    "The biases in dense and dense_2 introduce a slight positive bias to the activations in subsequent layers, potentially affecting the model's predictions. The biases in dense_1 and dense_3 are centered around zero, with a slight spread towards positive values, introducing a small positive shift in the activations of the next layer.\n",
    "\n",
    "The impact of these biases would depend on the network architecture and data. Overall, the model's weights and biases suggest that it is learning effectively from the training data."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "4114f5c851874555"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 1000x500 with 2 Axes>",
      "image/png": "[encoded data, 28740 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 1000x500 with 2 Axes>",
      "image/png": "[encoded data, 23456 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 1000x500 with 2 Axes>",
      "image/png": "[encoded data, 30932 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": "<Figure size 1000x500 with 2 Axes>",
      "image/png": "[encoded data, 24348 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 1. Visualize Weights and Biases\n",
    "for layer in model.layers:\n",
    "    if 'dense' in layer.name:\n",
    "        weights, biases = layer.get_weights()\n",
    "        plt.figure(figsize=(10, 5))\n",
    "        plt.subplot(1, 2, 1)\n",
    "        plt.hist(weights.flatten())\n",
    "        plt.title(f'{layer.name} weights')\n",
    "        plt.subplot(1, 2, 2)\n",
    "        plt.hist(biases.flatten())\n",
    "        plt.title(f'{layer.name} biases')\n",
    "        plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:49:35.361384Z",
     "start_time": "2024-03-20T11:49:34.075287Z"
    }
   },
   "id": "41091791008ff727",
   "execution_count": 111
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Confusion Matrix Evaluation\n",
    "\n",
    "The confusion matrix is a table that is often used to describe the performance of a classification model on a set of test data for which the true values are known. It gives a more detailed breakdown of correct and incorrect classifications for each class.\n",
    "\n",
    "The confusion matrix shows that the model has a good performance in distinguishing between positive and negative instances. The majority of the instances are correctly classified, with a small number of false positives and false negatives."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "618bc6deb5ea296b"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 500x500 with 2 Axes>",
      "image/png": "[encoded data, 26204 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 2. Confusion Matrix\n",
    "# Convert the predicted probabilities to binary outputs\n",
    "y_pred_classes = (y_pred > 0.5).astype(\"int32\")\n",
    "# Generate the confusion matrix\n",
    "cm = confusion_matrix(y_test, y_pred_classes)\n",
    "# Plot the confusion matrix\n",
    "plt.figure(figsize=(5, 5))\n",
    "sns.heatmap(cm, annot=True, fmt='d', cmap='Blues')\n",
    "plt.title('Confusion matrix')\n",
    "plt.xlabel('Predicted class')\n",
    "plt.ylabel('True class')\n",
    "plt.show()\n",
    "\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:49:35.531355Z",
     "start_time": "2024-03-20T11:49:35.362848Z"
    }
   },
   "id": "6b7d586ea49a858a",
   "execution_count": 112
  },
  {
   "cell_type": "markdown",
   "source": [
    "## ROC Curve Evaluation\n",
    "\n",
    "The Receiver Operating Characteristic (ROC) curve is a graphical representation that illustrates the performance of a binary classification model at all classification thresholds. It is commonly used in machine learning to evaluate the performance of a classifier system for two-class problems.\n",
    "\n",
    "The ROC curve has two axes:\n",
    "- The X-axis represents the False Positive Rate (FPR), which is the proportion of negative instances that are incorrectly classified as positive.\n",
    "- The Y-axis represents the True Positive Rate (TPR), which is the proportion of positive instances that are correctly classified as positive.\n",
    "\n",
    "A perfect classifier would classify all positive instances correctly (TPR = 1) and all negative instances correctly (FPR = 0). This would be represented by a curve that goes straight up the left side of the ROC graph and then along the top to the right corner.\n",
    "\n",
    "The Area Under the Curve (AUC) is a numerical measure of the ROC curve’s performance. A larger AUC indicates a better performance. In our case, the AUC is 0.91, which is considered to be very good.\n",
    "\n",
    "In summary, the ROC curve shows that our binary classification model has a good performance in distinguishing between positive and negative instances."
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "8645f7e159d38f0a"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 640x480 with 1 Axes>",
      "image/png": "[encoded data, 47216 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 3. ROC Curve\n",
    "# Compute ROC curve and ROC area for each class\n",
    "fpr, tpr, _ = roc_curve(y_test, y_pred)\n",
    "roc_auc = auc(fpr, tpr)\n",
    "# Plot the ROC curve\n",
    "plt.figure()\n",
    "lw = 2\n",
    "plt.plot(fpr, tpr, color='darkorange', lw=lw, label='ROC curve (area = %0.2f)' % roc_auc)\n",
    "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver Operating Characteristic')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:49:35.715900Z",
     "start_time": "2024-03-20T11:49:35.532577Z"
    }
   },
   "id": "4d080bef0cf9bec4",
   "execution_count": 113
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Learning curve"
   ],
   "metadata": {
    "collapsed": false
   },
   "id": "5d1867cb3af9788d"
  },
  {
   "cell_type": "code",
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 1000x600 with 1 Axes>",
      "image/png": "[encoded data, 79356 characters]"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_learning_curve(history):\n",
    "    plt.figure(figsize=(10, 6))\n",
    "    plt.plot(history.history['loss'], label='Training Loss')\n",
    "    plt.plot(history.history['val_loss'], label='Validation Loss')\n",
    "    plt.plot(history.history['accuracy'], label='Training Accuracy')\n",
    "    plt.plot(history.history['val_accuracy'], label='Validation Accuracy')\n",
    "    plt.xlabel('Epoch')\n",
    "    plt.ylabel('Loss / Accuracy')\n",
    "    plt.title('Learning Curve')\n",
    "    plt.legend()\n",
    "    plt.show()\n",
    "\n",
    "plot_learning_curve(history)\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-03-20T11:49:35.981276Z",
     "start_time": "2024-03-20T11:49:35.717252Z"
    }
   },
   "id": "c67bb53e5a864293",
   "execution_count": 114
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
