GMM_vs_Spectral_Clustering
DownloadTélécharger
Actions
Vote :
ScreenshotAperçu
Informations
Catégorie :Category: nCreator TI-Nspire
Auteur Author: Dinho_Darroz
Type : Classeur 3.0.1
Page(s) : 1
Taille Size: 3.67 Ko KB
Mis en ligne Uploaded: 18/06/2024 - 08:42:19
Mis à jour Updated: 18/06/2024 - 08:42:23
Uploadeur Uploader: Dinho_Darroz (Profil)
Téléchargements Downloads: 4
Visibilité Visibility: Archive publique
Shortlink : https://tipla.net/a4080806
Type : Classeur 3.0.1
Page(s) : 1
Taille Size: 3.67 Ko KB
Mis en ligne Uploaded: 18/06/2024 - 08:42:19
Mis à jour Updated: 18/06/2024 - 08:42:23
Uploadeur Uploader: Dinho_Darroz (Profil)
Téléchargements Downloads: 4
Visibilité Visibility: Archive publique
Shortlink : https://tipla.net/a4080806
Description
Fichier Nspire généré sur TI-Planet.org.
Compatible OS 3.0 et ultérieurs.
<<
Gaussian Mixture Model (GMM) Overview A Gaussian Mixture Model (GMM) is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed dataset explicitly indicate which subpopulation each data point belongs to. It is commonly used for clustering. Key Concepts Mixture Model : A weighted sum of multiple Gaussian distributions. Latent Variables : Variables that indicate the subpopulation (or cluster) a data point belongs to. Expectation-Maximization (EM) Algorithm : An iterative method to find the maximum likelihood estimates of parameters in the presence of latent variables. GMM Components Gaussian Components : Each component is a Gaussian distribution defined by its mean vector ¼kmu_k ¼ k and covariance matrix £kSigma_k £ k . Mixing Coefficients : Weights Àkpi_k À k for each Gaussian component, representing the probability that a randomly selected data point belongs to the kk k -th Gaussian. Mathematical Formulation The probability density function of a GMM is: p(x)=k=1KÀkN(x#¼k,£k)p(x) = sum_{k=1}^K pi_k mathcal{N}(x | mu_k, Sigma_k) p ( x ) = k = 1 K À k N ( x # ¼ k , £ k ) where N(x#¼k,£k)mathcal{N}(x | mu_k, Sigma_k) N ( x # ¼ k , £ k ) is the Gaussian distribution with mean ¼kmu_k ¼ k and covariance £kSigma_k £ k . Expectation-Maximization (EM) Algorithm Initialization : Initialize the means ¼kmu_k ¼ k , covariances £kSigma_k £ k , and mixing coefficients Àkpi_k À k . Expectation (E) Step : Calculate the responsibilities ³ikgamma_{ik} ³ ik , which represent the probability that data point xix_i x i belongs to component kk k : ³ik=ÀkN(xi#¼k,£k)j=1KÀjN(xi#¼j,£j)gamma_{ik} = frac{pi_k mathcal{N}(x_i | mu_k, Sigma_k)}{sum_{j=1}^K pi_j mathcal{N}(x_i | mu_j, Sigma_j)} ³ ik = j = 1 K À j N ( x i # ¼ j , £ j ) À k N ( x i # ¼ k , £ k ) Maximization (M) Step : Update the parameters ¼kmu_k ¼ k , £kSigma_k £ k , and Àkpi_k À k : ¼k=i=1N³ikxii=1N³ikmu_k = frac{sum_{i=1}^N gamma_{ik} x_i}{sum_{i=1}^N gamma_{ik}} ¼ k = i = 1 N ³ ik i = 1 N ³ ik x i £k=i=1N³ik(xi¼k)(xi¼k)Ti=1N³ikSigma_k = frac{sum_{i=1}^N gamma_{ik} (x_i - mu_k)(x_i - mu_k)^T}{sum_{i=1}^N gamma_{ik}} £ k = i = 1 N ³ ik i = 1 N ³ ik ( x i ¼ k ) ( x i ¼ k ) T Àk=1Ni=1N³ikpi_k = frac{1}{N} sum_{i=1}^N gamma_{ik} À k = N 1 i = 1 N ³ ik Convergence Check : Check for convergence (e.g., change in log-likelihood below a threshold). Applications Clustering Density estimation Anomaly detection Spectral Clustering Overview Spectral clustering is a graph-based clustering method that uses the eigenvalues (spectrum) of a similarity matrix to reduce dimensionality before clustering in fewer dimensions. Key Concepts Graph Representation : Data points are represented as nodes, and edges represent similarities between points. Laplacian Matrix : A matrix representing the graph, defined as L=DWL = D - W L = D W , where WW W is the similarity matrix and DD D is the degree matrix. Eigenvalues and Eigenvectors : Used to capture the essential structure of the data. Spectral Clustering Steps Construct Similarity Matrix WW W : Calculate the similarity between each pair of data points. Common choices include Gaussian (RBF) kernel. Construct Laplacian Matrix LL L : L=DWL = D - W L = D W where DD D is the diagonal degree matrix. Compute Eigenvalues and Eigenvectors : Calculate the first kk k eigenvalues and their corresponding eigenvectors of the Laplacian matrix LL L . Form Matrix UU U : Form a matrix UU U with the eigenvectors as columns. Normalize Rows : Normalize each row of UU U to have unit length. Clustering : Treat each row of UU U as a point in Rkmathbb{R}^k R k and cluster them using K-means or another clustering algorithm. Applications Image segmentation Social network analysis Document clustering Comparison and Relation GMM vs. Spectral Clustering : GMM is a model-based clustering technique that assumes the data is generated from a mixture of several Gaussian distributions. Spectral clustering is a graph-based method that uses eigenvalues and eigenvectors of a similarity matrix to perform clustering. Commonality : Both methods aim to find natural groupings in the data, but they use different approaches and assumptions. Made with nCreator - tiplanet.org
>>
Compatible OS 3.0 et ultérieurs.
<<
Gaussian Mixture Model (GMM) Overview A Gaussian Mixture Model (GMM) is a probabilistic model for representing the presence of subpopulations within an overall population, without requiring that an observed dataset explicitly indicate which subpopulation each data point belongs to. It is commonly used for clustering. Key Concepts Mixture Model : A weighted sum of multiple Gaussian distributions. Latent Variables : Variables that indicate the subpopulation (or cluster) a data point belongs to. Expectation-Maximization (EM) Algorithm : An iterative method to find the maximum likelihood estimates of parameters in the presence of latent variables. GMM Components Gaussian Components : Each component is a Gaussian distribution defined by its mean vector ¼kmu_k ¼ k and covariance matrix £kSigma_k £ k . Mixing Coefficients : Weights Àkpi_k À k for each Gaussian component, representing the probability that a randomly selected data point belongs to the kk k -th Gaussian. Mathematical Formulation The probability density function of a GMM is: p(x)=k=1KÀkN(x#¼k,£k)p(x) = sum_{k=1}^K pi_k mathcal{N}(x | mu_k, Sigma_k) p ( x ) = k = 1 K À k N ( x # ¼ k , £ k ) where N(x#¼k,£k)mathcal{N}(x | mu_k, Sigma_k) N ( x # ¼ k , £ k ) is the Gaussian distribution with mean ¼kmu_k ¼ k and covariance £kSigma_k £ k . Expectation-Maximization (EM) Algorithm Initialization : Initialize the means ¼kmu_k ¼ k , covariances £kSigma_k £ k , and mixing coefficients Àkpi_k À k . Expectation (E) Step : Calculate the responsibilities ³ikgamma_{ik} ³ ik , which represent the probability that data point xix_i x i belongs to component kk k : ³ik=ÀkN(xi#¼k,£k)j=1KÀjN(xi#¼j,£j)gamma_{ik} = frac{pi_k mathcal{N}(x_i | mu_k, Sigma_k)}{sum_{j=1}^K pi_j mathcal{N}(x_i | mu_j, Sigma_j)} ³ ik = j = 1 K À j N ( x i # ¼ j , £ j ) À k N ( x i # ¼ k , £ k ) Maximization (M) Step : Update the parameters ¼kmu_k ¼ k , £kSigma_k £ k , and Àkpi_k À k : ¼k=i=1N³ikxii=1N³ikmu_k = frac{sum_{i=1}^N gamma_{ik} x_i}{sum_{i=1}^N gamma_{ik}} ¼ k = i = 1 N ³ ik i = 1 N ³ ik x i £k=i=1N³ik(xi¼k)(xi¼k)Ti=1N³ikSigma_k = frac{sum_{i=1}^N gamma_{ik} (x_i - mu_k)(x_i - mu_k)^T}{sum_{i=1}^N gamma_{ik}} £ k = i = 1 N ³ ik i = 1 N ³ ik ( x i ¼ k ) ( x i ¼ k ) T Àk=1Ni=1N³ikpi_k = frac{1}{N} sum_{i=1}^N gamma_{ik} À k = N 1 i = 1 N ³ ik Convergence Check : Check for convergence (e.g., change in log-likelihood below a threshold). Applications Clustering Density estimation Anomaly detection Spectral Clustering Overview Spectral clustering is a graph-based clustering method that uses the eigenvalues (spectrum) of a similarity matrix to reduce dimensionality before clustering in fewer dimensions. Key Concepts Graph Representation : Data points are represented as nodes, and edges represent similarities between points. Laplacian Matrix : A matrix representing the graph, defined as L=DWL = D - W L = D W , where WW W is the similarity matrix and DD D is the degree matrix. Eigenvalues and Eigenvectors : Used to capture the essential structure of the data. Spectral Clustering Steps Construct Similarity Matrix WW W : Calculate the similarity between each pair of data points. Common choices include Gaussian (RBF) kernel. Construct Laplacian Matrix LL L : L=DWL = D - W L = D W where DD D is the diagonal degree matrix. Compute Eigenvalues and Eigenvectors : Calculate the first kk k eigenvalues and their corresponding eigenvectors of the Laplacian matrix LL L . Form Matrix UU U : Form a matrix UU U with the eigenvectors as columns. Normalize Rows : Normalize each row of UU U to have unit length. Clustering : Treat each row of UU U as a point in Rkmathbb{R}^k R k and cluster them using K-means or another clustering algorithm. Applications Image segmentation Social network analysis Document clustering Comparison and Relation GMM vs. Spectral Clustering : GMM is a model-based clustering technique that assumes the data is generated from a mixture of several Gaussian distributions. Spectral clustering is a graph-based method that uses eigenvalues and eigenvectors of a similarity matrix to perform clustering. Commonality : Both methods aim to find natural groupings in the data, but they use different approaches and assumptions. Made with nCreator - tiplanet.org
>>
