KalCal: a novel calibration framework for radio interferometry using the Kalman Filter and Smoother
- Authors: Welman, Brian Allister
- Date: 2024-10-11
- Subjects: Radio interferometers , Calibration , Kalman filtering , Bayesian inference , Signal processing , Radio astronomy , MeerKAT
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/467127 , vital:76818
- Description: Calibration in radio interferometry is essential for correcting measurement errors. Traditional methods employ maximum likelihood techniques and non-linear least squares solvers but face challenges due to the data volumes and increased noise sensitivity of contemporary instruments such as MeerKAT. A common approach for mitigating these issues is using “solution intervals”, which helps manage the data volume and reduces overfitting. However, inappropriate interval sizes can degrade calibration quality, and determining optimal sizes is challenging, often relying on brute-force methods. This study introduces Kalman Filtering and Smoothing in Calibration (KalCal), a new framework for calibration that combines the Kalman Filter, Kalman Smoother, and the energy function: the negative logarithm of the Bayesian evidence. KalCal offers Bayesian-optimal solutions as probability densities and models calibration effects with lower computational requirements than iterative approaches. Unlike traditional methods, which require all the data for a particular solution to be in memory simultaneously, KalCal’s recursive computations only need a single pass through the data with appropriate prior information. The energy function provides the means for KalCal to determine this prior information. Theoretical contributions include additions to complex optimisation literature and the “Kalman-Woodbury Identity” that reformulates the traditional Kalman Filter. A Python implementation of the KalCal framework was benchmarked against solution intervals as implemented in the QuartiCal package. Simulations show KalCal matching solution intervals in high Signal-to-Noise Ratio (SNR) scenarios and surpassing them in low SNR conditions. Moreover, the energy function produced minima that coincide with KalCal’s Mean Square Error (MSE) on the true gain signal. This result is significant as the MSE is unavailable in real applications. Further research is needed to assess the computational feasibility and intricacies of KalCal. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2024
- Full Text:
- Date Issued: 2024-10-11
- Authors: Welman, Brian Allister
- Date: 2024-10-11
- Subjects: Radio interferometers , Calibration , Kalman filtering , Bayesian inference , Signal processing , Radio astronomy , MeerKAT
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/467127 , vital:76818
- Description: Calibration in radio interferometry is essential for correcting measurement errors. Traditional methods employ maximum likelihood techniques and non-linear least squares solvers but face challenges due to the data volumes and increased noise sensitivity of contemporary instruments such as MeerKAT. A common approach for mitigating these issues is using “solution intervals”, which helps manage the data volume and reduces overfitting. However, inappropriate interval sizes can degrade calibration quality, and determining optimal sizes is challenging, often relying on brute-force methods. This study introduces Kalman Filtering and Smoothing in Calibration (KalCal), a new framework for calibration that combines the Kalman Filter, Kalman Smoother, and the energy function: the negative logarithm of the Bayesian evidence. KalCal offers Bayesian-optimal solutions as probability densities and models calibration effects with lower computational requirements than iterative approaches. Unlike traditional methods, which require all the data for a particular solution to be in memory simultaneously, KalCal’s recursive computations only need a single pass through the data with appropriate prior information. The energy function provides the means for KalCal to determine this prior information. Theoretical contributions include additions to complex optimisation literature and the “Kalman-Woodbury Identity” that reformulates the traditional Kalman Filter. A Python implementation of the KalCal framework was benchmarked against solution intervals as implemented in the QuartiCal package. Simulations show KalCal matching solution intervals in high Signal-to-Noise Ratio (SNR) scenarios and surpassing them in low SNR conditions. Moreover, the energy function produced minima that coincide with KalCal’s Mean Square Error (MSE) on the true gain signal. This result is significant as the MSE is unavailable in real applications. Further research is needed to assess the computational feasibility and intricacies of KalCal. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2024
- Full Text:
- Date Issued: 2024-10-11
MeerKAT observations of three high-redshift galaxy clusters
- Authors: Manaka, Sinah Mokatako
- Date: 2023-03-29
- Subjects: MeerKAT , Galaxies Clusters , Calibration , Radio interferometers , Radio halo
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/422367 , vital:71936
- Description: Galaxy clusters are the largest, gravitationally-bound structures in the Universe, formed through the hierarchical merger of smaller structures. The most accepted view is that the merging process injects energy into the intracluster medium (ICM) and re-accelerates pre-existing particles and compresses magnetic fields, generating, eventually, synchrotron emission. Such radio emission appears as radio halos, i.e. central Mpc-size diffuse structures, mostly visible in merging or unrelaxed clusters and with a spatial correspondence with the thermal gas component of the ICM. Observations have probed radio halo properties mostly for clusters withM500 > 6×1014 M⊙ at intermediate redshifts (0.3 < z < 0.4), providing support to their connection between mergers, which provide the necessary energy to re-accelerate particles via turbulence. Probing the redshift evolution of radio halos is an important test of the turbulent re-acceleration scenario, as fewer halos are expected at high redshift, given the same mass interval. In this thesis, we present MeerKAT observations at 1.28 GHz of three high-redshift (PSZ2G254.08- 58.45, PSZ2G255.60-46.18 and PSZ2G277.76-51.74, in the 0.42 ≲ z ≲ 0.46 range) clusters, with masses M500 ≳ 6.2 × 1014 M⊙, selected for their disturbed dynamical state – inferred from existing X-ray observations. Our observations reached rms noise values between 20 and 23.5 μJy beam−1, with ∼ 4′′ angular resolution. No evidence of diffuse emission is found at ii full resolution. Low-resolution (∼ 30′′) images achieved rms noise levels of 30-50 μJy beam−1, amongst the deepest observations of high-redshift targets. One radio halo was detected in the least massive cluster PSZ2G254.08-58.45 extending over ∼ 500 kpc, with a 1.20 } 0.08 mJy integrated flux density. We placed a ∼1 mJy upper limit at 95% confidence level on the radio halo flux density for the other two targets. The radio-halo detection is consistent with the recent P1.4 GHz − M500 correlation from Cuciti et al. (2021b), while the upper limit on PSZ2G255.60-46.18 is consistent with being on the correlation. On the other hand, the upper limit on PSZ2G277.76-51.74 places the radio halo well below the correlation. Recently a 1.5 GHz survey (Giovannini et al., 2020) detected a slightly higher fraction of radio halos in clusters in the same redshift range, with power and size typically higher than what we found in our observations. Both observations are, however, not inconsistent with each other. Our results, although with limited statistics, do not disfavour the current scenario of radiohalo formation based on the turbulent re-acceleration model. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2023
- Full Text:
- Date Issued: 2023-03-29
- Authors: Manaka, Sinah Mokatako
- Date: 2023-03-29
- Subjects: MeerKAT , Galaxies Clusters , Calibration , Radio interferometers , Radio halo
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/422367 , vital:71936
- Description: Galaxy clusters are the largest, gravitationally-bound structures in the Universe, formed through the hierarchical merger of smaller structures. The most accepted view is that the merging process injects energy into the intracluster medium (ICM) and re-accelerates pre-existing particles and compresses magnetic fields, generating, eventually, synchrotron emission. Such radio emission appears as radio halos, i.e. central Mpc-size diffuse structures, mostly visible in merging or unrelaxed clusters and with a spatial correspondence with the thermal gas component of the ICM. Observations have probed radio halo properties mostly for clusters withM500 > 6×1014 M⊙ at intermediate redshifts (0.3 < z < 0.4), providing support to their connection between mergers, which provide the necessary energy to re-accelerate particles via turbulence. Probing the redshift evolution of radio halos is an important test of the turbulent re-acceleration scenario, as fewer halos are expected at high redshift, given the same mass interval. In this thesis, we present MeerKAT observations at 1.28 GHz of three high-redshift (PSZ2G254.08- 58.45, PSZ2G255.60-46.18 and PSZ2G277.76-51.74, in the 0.42 ≲ z ≲ 0.46 range) clusters, with masses M500 ≳ 6.2 × 1014 M⊙, selected for their disturbed dynamical state – inferred from existing X-ray observations. Our observations reached rms noise values between 20 and 23.5 μJy beam−1, with ∼ 4′′ angular resolution. No evidence of diffuse emission is found at ii full resolution. Low-resolution (∼ 30′′) images achieved rms noise levels of 30-50 μJy beam−1, amongst the deepest observations of high-redshift targets. One radio halo was detected in the least massive cluster PSZ2G254.08-58.45 extending over ∼ 500 kpc, with a 1.20 } 0.08 mJy integrated flux density. We placed a ∼1 mJy upper limit at 95% confidence level on the radio halo flux density for the other two targets. The radio-halo detection is consistent with the recent P1.4 GHz − M500 correlation from Cuciti et al. (2021b), while the upper limit on PSZ2G255.60-46.18 is consistent with being on the correlation. On the other hand, the upper limit on PSZ2G277.76-51.74 places the radio halo well below the correlation. Recently a 1.5 GHz survey (Giovannini et al., 2020) detected a slightly higher fraction of radio halos in clusters in the same redshift range, with power and size typically higher than what we found in our observations. Both observations are, however, not inconsistent with each other. Our results, although with limited statistics, do not disfavour the current scenario of radiohalo formation based on the turbulent re-acceleration model. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2023
- Full Text:
- Date Issued: 2023-03-29
Third generation calibrations for Meerkat Observation of Saraswati Supercluster
- Authors: Kincaid, Robert Daniel
- Date: 2022-10-14
- Subjects: Square Kilometre Array (Project) , Superclusters , Saraswati Supercluster , Radio astronomy , MeerKAT , Calibration
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/362916 , vital:65374
- Description: The international collaboration of the Square Kilometre Array (SKA), which is one of the largest and most challenging science projects of the 21st century, will bring a revolution in radio astronomy in terms of sensitivity and resolution. The recent launch of several new radio instruments, combined with the subsequent developments in calibration and imaging techniques, has dramatically advanced this field over the past few years, thus enhancing our knowledge of the radio universe. Various SKA pathfinders around the world have been developed (and more are planned for construction) that have laid down a firm foundation for the SKA in terms of science while additionally giving insight into the technological requirements required for the projected data outputs to become manageable. South Africa has recently built the new MeerKAT telescope, which is a SKA precursor forming an integral part of SKA-mid component. The MeerKAT instrument has unprecedented sensitivity that can cater for the required science goals of the current and future SKA era. It is noticeable from MeerKAT and other precursors that the data produced by these instruments are significantly challenging to calibrate and image. Calibration-related artefacts intrinsic to bright sources are of major concern since, they limit the Dynamic Range (DR) and image fidelity of the resulting images and cause flux suppression of extended sources. Diffuse radio sources from galaxy clusters in the form of halos, relics and most recently bridges on the Mpc scale, because of their diffuse nature combined with wide field of view (FoV) observations, make them particularly good candidates for testing the different approaches of calibration. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2022
- Full Text:
- Date Issued: 2022-10-14
- Authors: Kincaid, Robert Daniel
- Date: 2022-10-14
- Subjects: Square Kilometre Array (Project) , Superclusters , Saraswati Supercluster , Radio astronomy , MeerKAT , Calibration
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/362916 , vital:65374
- Description: The international collaboration of the Square Kilometre Array (SKA), which is one of the largest and most challenging science projects of the 21st century, will bring a revolution in radio astronomy in terms of sensitivity and resolution. The recent launch of several new radio instruments, combined with the subsequent developments in calibration and imaging techniques, has dramatically advanced this field over the past few years, thus enhancing our knowledge of the radio universe. Various SKA pathfinders around the world have been developed (and more are planned for construction) that have laid down a firm foundation for the SKA in terms of science while additionally giving insight into the technological requirements required for the projected data outputs to become manageable. South Africa has recently built the new MeerKAT telescope, which is a SKA precursor forming an integral part of SKA-mid component. The MeerKAT instrument has unprecedented sensitivity that can cater for the required science goals of the current and future SKA era. It is noticeable from MeerKAT and other precursors that the data produced by these instruments are significantly challenging to calibrate and image. Calibration-related artefacts intrinsic to bright sources are of major concern since, they limit the Dynamic Range (DR) and image fidelity of the resulting images and cause flux suppression of extended sources. Diffuse radio sources from galaxy clusters in the form of halos, relics and most recently bridges on the Mpc scale, because of their diffuse nature combined with wide field of view (FoV) observations, make them particularly good candidates for testing the different approaches of calibration. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2022
- Full Text:
- Date Issued: 2022-10-14
Machine learning methods for calibrating radio interferometric data
- Authors: Zitha, Simphiwe Nhlanhla
- Date: 2019
- Subjects: Calibration , Radio astronomy -- Data processing , Radio astronomy -- South Africa , Karoo Array Telescope (South Africa) , Radio telescopes -- South Africa , Common Astronomy Software Application (Computer software)
- Language: English
- Type: text , Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10962/97096 , vital:31398
- Description: The applications of machine learning have created an opportunity to deal with complex problems currently encountered in radio astronomy data processing. Calibration is one of the most important data processing steps required to produce high dynamic range images. This process involves the determination of calibration parameters, both instrumental and astronomical, to correct the collected data. Typically, astronomers use a package such as Common Astronomy Software Applications (CASA) to compute the gain solutions based on regular observations of a known calibrator source. In this work we present applications of machine learning to first generation calibration (1GC), using the KAT-7 telescope environmental and pointing sensor data recorded during observations. Applying machine learning to 1GC, as opposed to calculating the gain solutions in CASA, has shown evidence of reducing computation, as well as accurately predict the 1GC gain solutions representing the behaviour of the antenna during an observation. These methods are computationally less expensive, however they have not fully learned to generalise in predicting accurate 1GC solutions by looking at environmental and pointing sensors. We call this multi-output regression model ZCal, which is based on random forest, decision trees, extremely randomized trees and K-nearest neighbor algorithms. The prediction error obtained during the testing of our model on testing data is ≈ 0.01 < rmse < 0.09 for gain amplitude per antenna, and 0.2 rad < rmse <0.5 rad for gain phase. This shows that the instrumental parameters used to train our model more strongly correlate with gain amplitude effects than phase.
- Full Text:
- Date Issued: 2019
- Authors: Zitha, Simphiwe Nhlanhla
- Date: 2019
- Subjects: Calibration , Radio astronomy -- Data processing , Radio astronomy -- South Africa , Karoo Array Telescope (South Africa) , Radio telescopes -- South Africa , Common Astronomy Software Application (Computer software)
- Language: English
- Type: text , Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10962/97096 , vital:31398
- Description: The applications of machine learning have created an opportunity to deal with complex problems currently encountered in radio astronomy data processing. Calibration is one of the most important data processing steps required to produce high dynamic range images. This process involves the determination of calibration parameters, both instrumental and astronomical, to correct the collected data. Typically, astronomers use a package such as Common Astronomy Software Applications (CASA) to compute the gain solutions based on regular observations of a known calibrator source. In this work we present applications of machine learning to first generation calibration (1GC), using the KAT-7 telescope environmental and pointing sensor data recorded during observations. Applying machine learning to 1GC, as opposed to calculating the gain solutions in CASA, has shown evidence of reducing computation, as well as accurately predict the 1GC gain solutions representing the behaviour of the antenna during an observation. These methods are computationally less expensive, however they have not fully learned to generalise in predicting accurate 1GC solutions by looking at environmental and pointing sensors. We call this multi-output regression model ZCal, which is based on random forest, decision trees, extremely randomized trees and K-nearest neighbor algorithms. The prediction error obtained during the testing of our model on testing data is ≈ 0.01 < rmse < 0.09 for gain amplitude per antenna, and 0.2 rad < rmse <0.5 rad for gain phase. This shows that the instrumental parameters used to train our model more strongly correlate with gain amplitude effects than phase.
- Full Text:
- Date Issued: 2019
Link between ghost artefacts, source suppression and incomplete calibration sky models
- Authors: Nunhokee, Chuneeta Devi
- Date: 2015
- Subjects: Interferometry , Calibration
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:5556 , http://hdl.handle.net/10962/d1017900
- Description: Calibration is a fundamental step towards producing radio interferometric images. However, naive calibration produces calibration artefacts, in the guise of spurious emission, buried in the thermal noise. This work investigates these calibration artefacts, henceforth referred to as “ghosts”. A 21 cm observation with the Westerbork Synthesis Radio Telescope yielded similar ghost sources, and it was anticipated that they were due to calibrating with incomplete sky models. An analytical ghost distribution of a two-source scenario is derived to substantiate this theory and to seek answers to the related bewildering features (regular ghost pattern, points spread function-like sidelobes, independent of model flux). The theoretically predicted ghost distribution qualitatively matches with the observational ones and shows high dependence on the array geometry. The theory draws the conclusion that both the ghost phenomenon and suppression of the unmodelled flux have the same root cause. In addition, the suppression of the unmodelled flux is studied as functions of unmodelled flux, differential gain solution interval and the number of sources subjected to direction-dependent gains. These studies summarise that the suppression rate is constant irrespective of the degree of incompleteness of the calibration sky model. In the presence of a direction-dependent effect, the suppression drastically increases; however, this increase can be compensated for by using longer solution intervals.
- Full Text:
- Date Issued: 2015
- Authors: Nunhokee, Chuneeta Devi
- Date: 2015
- Subjects: Interferometry , Calibration
- Language: English
- Type: Thesis , Masters , MSc
- Identifier: vital:5556 , http://hdl.handle.net/10962/d1017900
- Description: Calibration is a fundamental step towards producing radio interferometric images. However, naive calibration produces calibration artefacts, in the guise of spurious emission, buried in the thermal noise. This work investigates these calibration artefacts, henceforth referred to as “ghosts”. A 21 cm observation with the Westerbork Synthesis Radio Telescope yielded similar ghost sources, and it was anticipated that they were due to calibrating with incomplete sky models. An analytical ghost distribution of a two-source scenario is derived to substantiate this theory and to seek answers to the related bewildering features (regular ghost pattern, points spread function-like sidelobes, independent of model flux). The theoretically predicted ghost distribution qualitatively matches with the observational ones and shows high dependence on the array geometry. The theory draws the conclusion that both the ghost phenomenon and suppression of the unmodelled flux have the same root cause. In addition, the suppression of the unmodelled flux is studied as functions of unmodelled flux, differential gain solution interval and the number of sources subjected to direction-dependent gains. These studies summarise that the suppression rate is constant irrespective of the degree of incompleteness of the calibration sky model. In the presence of a direction-dependent effect, the suppression drastically increases; however, this increase can be compensated for by using longer solution intervals.
- Full Text:
- Date Issued: 2015
- «
- ‹
- 1
- ›
- »