Purchase Solution

Anomaly detection

Not what you're looking for?

Ask Custom Question

This will explore PCA models for anomaly detection and diagnosis. Data is available on tab 1, 2 and 3 on the excel file attached, which includes three matrices - data, test1, and test2 - with 21 variables each. The matrices contain data collected from a Slurry Red Ceramic Melter (SRCM): the first twenty variables are temperature readings from thermowells throughout the Melter and the 21st variable is the glass level. The temperature readings are in units of degree Celsius(°C), while the glass level reading is in inches. Here, we will use principal component analysis to determine if the data included in test1 and test2 matches with the data in data.

Q1. Briefly describe PCA and how it is used for anomaly detection.
a. Describe the use of Q- and T2 -statistics for detecting anomalies.
b. Give the equations for calculating the Q- and T2 -statistics.

Q2. Develop a PCA model of the normal operating data contained in data.
a. Determine on your own how many PCs should be included in the model.
b. Describe how you made this decision and why it makes sense!
c. Using this model and the fault-free data in data, determine thresholds for the Q- and T2 -statistics. Explain how you selected thresholds.

Q3. Project the data in test1 and test2 into the PC space that you found in Q2. Evaluate the T2
- and Q-statistics to identify which, if any, observations are anomalous.

Q4. For data observations that are flagged as anomalies, look at the contributions to the T2 - or Q-statistics (whichever is appropriate) to determine which sensor readings might be causing the anomalies. Visually inspect the raw signals for these to see if the faults are apparent.

Attachments
Purchase this Solution
Solution Summary

The Solution explains the PCA models and anomaly detection and uses the data to prepare a sample model.

Solution provided by:
Chris Busu, PhD (IP)
Education
  • BSc, University of Bucharest
  • MSc, Ovidius
  • MSc, Stony Brook
  • PhD (IP), Stony Brook
Recent Feedback
  • "Thank you "
  • "Thank You Chris this draft really helped me understand correlation."
  • "Thanks for the prompt return. Going into the last meeting tonight before submission. "
  • "Thank you for your promptness and great work. This will serve as a great guideline to assist with the completion of our project."
  • "Thanks for the product. It is an excellent guideline for the group. "
Purchase this Solution
Free BrainMass Quizzes
Measures of Central Tendency

Tests knowledge of the three main measures of central tendency, including some simple calculation questions.

Terms and Definitions for Statistics

This quiz covers basic terms and definitions of statistics.

Measures of Central Tendency

This quiz evaluates the students understanding of the measures of central tendency seen in statistics. This quiz is specifically designed to incorporate the measures of central tendency as they relate to psychological research.

Know Your Statistical Concepts

Each question is a choice-summary multiple choice question that presents you with a statistical concept and then 4 numbered statements. You must decide which (if any) of the numbered statements is/are true as they relate to the statistical concept.

View More Free Quizzes
Related BrainMass Solutions

  • How data mining can be used to detect fraud ?

    The implicit assumption of anomaly based fraud detection is that anomaly events occur very rarely than normal data such that there characteristics are different from normality.

  • Intrusion

    280830 Describe the two main classes of intrusion Describe the differences between the two main classes of intrusions: misuse and anomaly. What are the characteristics of a good intrusion detection system?

  • Data Mining Patterns

    detection)- This involves detection of unusual data records which may require further investigation. 3. dependencies (association rule mining)- There is a dependence between different variables.

  • Essay - How to Secure Network Resources

    Install a network and host Intrusion Detection System (IDS) which can detect both misuse and anomaly based attacks. These are guidelines and not an outline.

  • Article Summary from the June 2011 issue of The Economist

    "TaKaDu" a firms based in Israel "developed a monitoring system called a statistical anomaly detection engine that is intended to identify clues in the data which might otherwise be missed". The system is operational in Jerusalem and London.

  • IT Architecutre: Basic Question & Answers

    What are the two types of detection that an IDS can provide? A) Misuse and anomaly B) Denial of service detection and HTTP attacks C) Fragment and offset D) ICMP and TCP 8.

  • Lethal genes in rabbit - pelger anomaly

    8501 Lethal genes in rabbit - pelger anomaly Pelger anomaly in rabbits involves white blood cell nuclear segmentation.

  • Important information about Networks

    Statistical Analysis This approach is widely used in the anomaly detection. Deviations from the mean value (that is dispersions) of the normal behavior profile notify the administrator of the fact that the attack has been detected.

  • Global Warming Statistics

    452911 Global Warming Statistics Below are Global carbon dioxide concentrations and the measured change in Global temperature (Temperature Anomaly) for an 11 year period in our recent past: YEAR CO2 Concentration (ppm) Temperature Anomaly (ºC)

View More Related Solutions