Purpose and design of the Hp indices, test dataset
The geomagnetic Hp indices are developed as part of the SWAMI project (http://swami-h2020.eu) funded by the European Union’s H2020 research and innovation program. They are designed to resemble the geomagnetic Kp index, but have a higher temporal resolution of 90, 60 and 30 minutes. Whereas the Kp index is a measure of energy input from the solar wind during a 3-hour interval, the Hp indices aim at being a similar measure for the energy input, but over shorter intervals. The geomagnetic Hp indices can be provided back to 1995. Their derivation procedure is similar, but not identical, to the Kp index. Hp values range from 0 to 9 (like Kp), and have mean occurrence rates that are comparable to those of the Kp index. However, users have to appreciate that the Hp indices are not identical to the Kp index of the corresponding time interval. Therefore, it is to be expected that they represent the energy input from the solar wind slightly differently than when using the Kp index.
Disclaimer to users of the Hp indices test data set
Please carefully test and validate all your model output and services for which you use the Hp indices (including the ap90, ap60, ap30) as input parameter. This is especially true when these models and services were originally derived or parameterized with the Kp index.
Which files to use?
We provide a number of test data files with different time resolutions. By default, we recommend to use the 1-hourly Kp-like Hp60 index (e.g. data file Hp60_2003.dat) or ap-like ap60 index (e.g. ap60_2003.dat).
Hp test dataset description
The Hp test dataset consists of 24 files. It is accompanied by the presentation given on the index at the IUGG General Assembly 2019 in Montreal (Stolle et al., 2019).
For each year 2003, 2004, 2005 and 2017, there exist annual files for 90, 60 and 30 minutes time resolution) in 2 different formats (Hp and ap). In the format 'Hp' the Hp values are given as 0, 0.7, 1, 1.3, 1.7, 2, 2.3, ... 8.7, 9. In the format 'ap', the Hp values are mapped onto ap values in the same fashion as Kp values are mapped to ap values.
The index is provided with an hourly resolution (Hp60 and ap60), and also with a 30-minute (Hp30 and ap30) and 90-minute version (Hp90 and ap90).
The years 2003 (Halloween storm in October and November), 2005 (frequent geomagnetic storms) and 2017 (geomagnetic storm in September) were chosen for the occurrence of strong geomagnetic activity.
The files are ASCII and have 7 header lines. The data is blank separated and fixed length. The 7th header line indicates the start time (in UTC) of the index interval. For Hp90 there are 16 intervals per day, for Hp60 there are 24 intervals per day, for Hp30 there are 48 intervals per day. Every line with data contains the index values for one day and starts with the date (year-month-day) in the format YYYY-MM-DD. The index values for each interval are written below the start time of the 7th header line. Missing data is indicated by -1.
For more information on the Kp and ap index, please refer to https://www.gfz-potsdam.de/en/kp-index/ and to Siebert and Meyer (1996). For more information on the Hp indices test dataset, please refer also to the presentation (Stolle et al., 2019) which can be downloaded from the FTP server.
The dataset contains hourly mean values (HMV) of the horizontal magnetic field component H as measured at the geomagnetic observatory Huancayo for 1958 to 2013. Huancayo observatory (IAGA code HUA) is operated by Instituto Geofisico del Peru. This dataset is based on and partly supersedes the data publication Matzka et al. (2017a), which is described in Matzka et al. (2017b). It will be superseded by another data publication (Soares et al., in preparation) that we intend to publish at GFZ Data Services as well as the World Data Centres for Geomagnetism.
As already described in Matzka et al. (2017a, 2017b), the HMVs were taken from the World Data Centre Kyoto (WDC Kyoto) and data gaps (in total some 19 years from the 1960ies, 1970ies and 1980ies) were filled in by typing handwritten records of the HMV at GFZ. These handwritten records were monthly tables that were received as digital images from geomagnetic observatory Huancayo or that were received as microfilms from World Data Centre Boulder. We also produced digital images of these microfilms. The values from the WDC Kyoto are definitive values; the monthly tables presumably also contain definitive values. Corrections to HUA HMVs from WDC Kyoto: There is a known error in the time stamping of the HUA HMVs prior to 1948 (before 1948 the data was reported in local time, rather than universal time). This error is corrected in the present dataset. Also, an attempt was made to correct for a jump in the HMV time series at this time.
Further corrections, made to the dataset by Matzka et al. (2017a), are mostly the correction or deletion of outliers and the correction of shifts in the data.
Again, please note that a dataset based on the data provided here will be submitted to the World Data Centres for Geomagnetism at a later stage and will have some additional modifications (Soares et al., in preparation).
The data file is in ASCII format and contains blank-separated first the year (YYYY), the month (MM), the day (DD) followed by the 24 HMVs of H (format HHHHH) in nanotesla (nT), starting with the HMV for 00 to 01 universal time. Geomagnetic observatories are described in e.g. Jankowski and Sucksdorf (1996), in Matzka et al., (2010) and Matzka (2016).
The HMC (Hourly Magnetospheric Currents) index measures the activity of large-scale magnetospheric currents on Earth's surface from 1900 to 2015. It resolves the absolute intensity of low-frequency variations, especially at periods relevant to the solar cycle, more robustly than existing geomagnetic indices. HMC is based on hourly means of vector magnetic field measurements from 34 mid latitude geomagnetic observatories obtained from WDC Edinburgh (http://www.wdc.bgs.ac.uk/catalog/master.html). This data has been manually revised to correct for spikes, jumps and drifts. A detailed description of the derivation method is given in Pick et al. (JGR, 2019) to which these data are supplementary material.
This directory contains the HMC index (hmc1900phor.hor) and the modified observatory data that it is based on (data.zip). The index and the observatory data files are formatted in compliance with the IAGA-2002 ASCII exchange format (https://www.ngdc.noaa.gov/IAGA/vdat/IAGA2002/iaga2002format.html).
Individual file names are composed of:
[IAGA code of observatory] + [first active year during 1900-2015] + [p(provisional)] + [hor(hourly)] + [_mod(modified)].hor
Also included is information on how the data modifications (list in modifications.pdf) were applied (readme.txt).