Skip links and keyboard navigation

Frequently asked questions

Do you provide customised datasets?

No. All SILO data products are now available through our self-service website in a range of customisable formats.

How do I reset my password?

To reset your password, click on Forgotten your password? on the SILO login page.

Do I need an account, or can I log in using my home institution credentials?

If you are affiliated with an institution that is a member of the Australian Access Federation (AAF), you can log in using your existing credentials. Otherwise you will need a SILO account. As this system is being decommissioned on 31 March 2019, new accounts are no longer being created. If you don't already have an account, please use our original system.

Why can't I log in using AAF if I also have a SILO account?

You cannot log in via AAF if you have previously created a SILO account with the same email address that is associated with your AAF credentials.

Why do I need to provide my email address?

Your email address is used to notify you when the data you requested are ready for download, monitor your download limit, and contact you about the service. Your personal data are protected by our privacy policy.

My email address has changed. Can I change it in SILO?

No. Your email address is your unique SILO username. If you change your email address you will need to create a new SILO account.

Note: this does not apply to users accessing SILO using their AAF credentials.

What is an API key?

When requesting data via our API, you need to provide an API key so we can track the request.

How do I create an API key?

After logging in, hover over the icon in the navigation bar and select My API keys. You can have up to four keys.

What are the differences between point data and gridded data?

Point datasets are temporal datasets at a single location. In other words, they provide a time-series of data (usually daily time-step) at either a single grid cell or a single station.

Gridded datasets are spatial datasets for a given date. SILO grids cover the region 112°E to 154°E, 10°S to 44°S with resolution 0.05° longitude by 0.05° latitude (approximately 5 km × 5 km).

What is the difference between point data at grid points and station locations?

Station point datasets are a time series of data at a station location, consisting of station records which have been supplemented by interpolated estimates when observed data are missing. Station point datasets are available at approximately 18,700 station locations around Australia. These datasets were formerly known as SILO Patched Point datasets.

Grid point datasets are a time series of data at a grid point location consisting entirely of interpolated estimates. The data are taken from our gridded datasets and are available at any grid point over the land area of Australia (including some islands). The nominal grid location (where the interpolated surface is evaluated) is the centre of the corresponding grid cell. These datasets were formerly known as SILO Data Drill datasets.

Why does the "all data" predefined format change?

When new variables are added to SILO, they are appended as additional columns on the right hand side in the "all data" format.

What happened to Patched Point and Data Drill datasets?

Both datasets are still available, but as they can now be requested through the same interface, the distinction between them is no longer required. For further information, see the previous question.

Are point data interpolated or observed?

Point datasets at grid locations consist entirely of interpolated data. Point datasets at station locations contain observed data (when available) and interpolated data (when observed data are not available or do not pass Quality Assurance tests).

What tools can I use to view, convert or process NetCDF rasters?

There are many open source and commercial tools available for working with NetCDF rasters. You might find the following tools helpful:

NetCDF files can be converted to a wide range of GIS formats using GDAL.

Why can't I read SILO's NetCDF files?

SILO rasters are packed into NetCDF files containing all 365 (or 366) daily rasters for a single year (and similarly, all 12 monthly rasters in the case of monthly rainfall). However the NetCDF file for the current year (for a given variable) only contains rasters for January 1 up to the previous day (or month), even though the NetCDF metadata indicate rasters are available for all timeslices within the year.

For example, on January 4 the NetCDF file for daily rainfall will contain rasters for January 1, 2 and 3. Inspection of the metadata however will show the time dimension is 365:
> ncdump -h 2018.daily_rain.nc
netcdf 2018.daily_rain {
dimensions:
      lat = 681 ;
      lon = 841 ;
      time = UNLIMITED ; // (365 currently)

Some software packages (such as R and NCO) cannot handle files with this structure. To work around the issue you may be able to extract the desired timeslices into a new NetCDF file with the time dimension set to the true number of timeslices. For example, you could extract the rasters for January 1 - January 3 using ncks (part of the NCO package):
> ncks -v daily_rain -F -d time,1,3 2018.daily_rain.nc 20180101-20180103.daily_rain.nc

What time are observations made?

For most daily climate variables the observations are recorded at 9 am. To assist in understanding how SILO datasets are constructed, it may be useful to see how data are assigned to a given day:

Observation times
Why are evaporation data shifted to the day before?

Evaporation (class A pan) is measured at 9am. In normal circumstances a large proportion of the observed evaporation would have occurred throughout the daylight hours (after 9am) on the previous day. Consequently, SILO shifts evaporation data to the day before the observation was made.

Note: the Bureau of Meteorology shifts maximum temperature data to the previous day for similar reasons. SILO uses the shifted temperature data provided by the Bureau.

Why are some SILO evaporation data different from the raw data from the Bureau of Meteorology?

Class A evaporation pans are usually fitted with a bird screen to stop animals drinking out of the pan. The screen reduces evaporation by approximately seven percent (van Dijk, 1985). To account for this effect, SILO increases the observed evaporation at a given station by 7% on all days prior to the installation of a screen at that location.

Reference: van Dijk, M. H. (1985). Reduction in evaporation due to the bird screen used in the Australian class A pan evaporation network. Australian Meteorogical Magazine 33, pp. 181–183.

How are gridded rainfall datasets created?

Daily rainfall gridded datasets are derived from interpolated monthly rainfall by partitioning the monthly total onto individual days. Partitioning requires estimation of the daily distribution throughout the month. The distribution is obtained by direct interpolation of daily rainfall data throughout the month. At the end of the month, the interpolated monthly rainfall is then partitioned onto individual days according to the computed distribution.

For further information, please read the journal article which documents many of SILO's processes, and also our metadata.

Can I request point data for a large number of locations through the website?

You can select a maximum of 50 locations in a single request through the website.

How can I repeat a data request for the same set of stations (or grid points)?

If you wish to repeat a previous request or automate requests, we suggest you try our API rather than manually entering requests via the interactive web page.

SILO has data for 29-Feb-2000 but not for 29-Feb-1900, is this right?

Yes. Under the Gregorian calendar 2000 was a leap year, but 1900 wasn't. Normally if the year can be evenly divided by 100 it is not a leap year, however it is a leap year if it can be evenly divided by 400.

What elevations are used in the interpolations?

SILO uses a Digital Elevation Model (DEM) constructed from NASA's 30m Shuttle Radar Topography Mission (SRTM) datasets.

Elevation is used as an independent variable when interpolating most of the variables provided by SILO (for details, see our journal article). The interpolation uses the position (longitude, latitude and elevation) and value of each observation, where the elevation is that of the recording station. In most cases SILO uses the station elevation supplied by the Bureau of Meteorology. However SILO uses a 30m resolution DEM to verify the elevation value provided by the Bureau. If the supplied value differs by more than 50m from the DEM value, SILO will use the DEM value instead of the value provided by the Bureau. When constructing the output grid, the interpolated estimate is computed at the centre of each grid cell using a mean elevation that is intended to be representative of the entire 0.05° × 0.05° grid cell. The mean elevation is computed from the elevations of all 30m × 30m pixels in the SRTM DEM that lie within the 0.05° × 0.05° grid cell and which are not masked out (for example, over ocean).

How are the grid locations selected?

The nominal location for a given grid cell is the centre of the cell. For example, the value at 115.05° East, 34.00° South is intended to be representative of the area 115.025° - 115.075° East, 33.975° - 34.025° South.

You can request data for any grid cell that is not masked out. Grid cells over the ocean and some islands are masked out. If you request data at a location where the longitude or latitude is not a multiple of 0.05°, the location will be rounded to the nearest 0.05°. If you request data at a location that lies exactly on the edge of a grid cell, the longitude (or latitude) will be rounded up giving you data from the grid cell that is east of the cell edge (if the specified longitude lies on a cell edge), or north of the cell edge (if the specified latitude lies on a cell edge).

Can you provide interpolated data at resolutions higher than 0.05° × 0.05°?

No. The accuracy of the interpolated datasets is strongly dependent on the density of the input data (i.e. station density). In most regions the station density is not high enough to support higher resolution estimates.

Users should also note that the interpolated estimates are based on observations recorded in a standard Stephenson screen or equivalent, placed in an open and usually flat area. It does not represent the ground level climate of some areas (e.g. wooded systems) and does not address all the issues associated with slope and aspect. Users may need to implement their own microclimate adjustments.

How accurate are the interpolated data prior to 1957?

The number of stations recording climate variables significantly increased around 1957 – the International Geophysical Year. An anomaly interpolation technique is used to interpolate maximum and minimum temperature, radiation and vapour pressure for all years prior to 1957. The anomaly method is better able to interpolate sparse datasets than direct interpolation because much of the variance can be captured by a mean dataset constructed from a larger dataset.

The data quality throughout the pre-1957 period was examined in: Rayner, D.P., Moodie, K.B., Beswick, A.R., Clarkson, N.M., and Hutchinson, R.L. (2004), New Australian daily historical climate surfaces using CLIMARC. Queensland Department of Natural Resources, Mines and Energy Report QNRME04247.

Are SILO data quality checked?

SILO datasets are constructed from observational data collected by the Bureau of Meteorology. The Bureau has a quality assurance program which is progressively checking its observational collection. SILO does not use data which have been quality checked by the Bureau and classified as "wrong", "suspect" or "inconsistent with other known information". In addition, SILO implements a number of internal checks to identify data which may be erroneous. For example, SILO uses a "two-pass" interpolation technique to interpolate all variables except daily rainfall. Observed data are interpolated in a first pass and residuals computed for all data points. The residual is the difference between the observed and interpolated values. Data points with high residuals may be indicative of erroneous data and are excluded from a subsequent interpolation which generates the final surface from which the station-point datasets are constructed.

Is it possible that minimum temperatures are higher than maximum temperature?

No. Maximum temperature observations are shifted to the previous day (see above), so the observed maximum should always be lower than (or equal to) the minimum temperature.

Please note the interpolated rasters are checked to ensure that every pixel in the maximum temperature raster is greater than or equal to the corresponding pixel in the minimum temperature raster. If the maximum temperature at a given pixel is less than the minimum temperature at the corresponding pixel in the corresponding minimum temperature raster, the pixels (in both rasters) are assigned the mean of the original maximum and minimum temperatures. This situation can arise from: (i) interpolation error (typically "overshoot"); or (ii) errors in the observed data used in the interpolation.

Why are the observed data and interpolated data different?

The observed data for a given station should be similar to the corresponding interpolated data at the nearest grid cell. However differences can arise for several reasons:

  1. interpolated data are evaluated at the centre of the grid cell. If the station is a significant distance from the cell centre, and the interpolated surface exhibits a strong gradient in the area, there can be a significant difference between the value at the station and the cell centre. For example, under normal conditions the temperature usually decreases with elevation at approximately 5-7 °C/km. If the elevation at the cell centre is 500m higher than the station (quite possible in alpine regions), the temperature at the cell centre could be 2.5-3.0 °C lower than the temperature at the station.
  2. most of SILO's interpolated grids are constructed using a smoothing spline. If the input data are spatially homogeneous, the fitted surface will generally pass through (i.e. reproduce) the input data. However if the input data are highly variable or contain errors, the spline may smooth the data and consequently the fitted surface will not reproduce the input data in the affected area(s). Note: SILO uses kriging to interpolate daily and monthly rainfall. Kriging guarantees the input data are reproduced.
  3. SILO uses a "two-pass" interpolation technique to interpolate all variables except daily rainfall. Data rejected in the first pass are excluded from the dataset used to construct the interpolated grid in the second pass. Consequently, if a given datum has been rejected by SILO's interpolation system, the fitted surface may differ substantially from the observed datum at that location. Users should note that data rejected by the interpolation system are included in the point (station) datasets and can be identified by their source flag. Users wishing to exclude such observations can replace them with interpolated estimates by requesting the corresponding data at the nearest grid cell.
Are the data suitable for determining the rates of changes in temperature, rainfall, evaporation, etc., caused by climate change?

SILO data are not intended for use in climate change detection studies. Small changes caused by climate change can be easily eclipsed by changes resulting from instrumental biases and relocating recording stations. For climate change detection we recommend using the Bureau's ACORN-SAT and High-Quality datasets.

Can I mirror SILO's datasets?

The gridded datasets are stored on Amazon Web Services' Public Data repository in NetCDF format in an S3 bucket. The NetCDF files can be mirrored using the AWS command line utilities for working with S3 datasets. For example, the monthly rainfall rasters can be mirrored to your current directory using the sync command:
aws s3 sync s3://silo-open-data/annual/monthly_rain/ .

The point datasets cannot be easily mirrored because:

  • there are a large number of files as there is one file for each station location (approximately 18,700) and one file for each grid point location (approximately 290,000)
  • every file changes every day (data can change for a variety of reasons; see the next question below).

If you need to maintain a mirror of point datasets, please consider:

  • using our API (it is designed for repetitive or automated tasks).
  • mirroring the data for only the subset of locations that you need i.e. not all locations as that would require mirroring a large number of files as discussed above.
  • your monthly download limit. You may exceed your limit if you attempt to mirror a large number of datasets.
  • building your own point datasets from our gridded datasets. Point datasets at grid locations can be built by extracting the relevant pixel values from a time-series of gridded datasets. The gridded data can be efficiently downloaded because they are arranged in annual blocks, with each annual file containing all of the grids for the selected year and variable. Please note this approach is not possible if you are seeking point datasets at station locations.
  • an incremental approach. For example, you could occasionally download data for the entire time period required (e.g. every six months to capture major changes to the data), and frequently update only the most recent 3 months of each dataset (e.g. every week to capture nightly changes to the data).

For further information and examples, please see our mirroring page.

Please remember SILO data are provided free of charge under the Queensland Government's Open Data program. In addition to the operational cost of maintaining the system, SILO also pays data egress charges for the data downloaded by our clients. We therefore ask you to carefully consider your data requirements before downloading large volumes of point data.

Why do SILO data change?

SILO data are constantly evolving. The changes can be a result of changes in the raw data or changes in SILO methodology. Recent datasets grow rapidly as new observational data are added. The data are also subject to corrections and updates by the Bureau, so there can be ongoing changes to the raw data used to construct SILO datasets.

SILO actively seeks techniques for improving data quality and may implement changes which modify our interpolated estimates or derived variables. Users requiring static datasets (e.g. for model calibration) should archive their own copy of the data, as they should not rely on SILO supplying exactly the same dataset at different points in time.

When does the nightly update occur?

SILO is updated every night with new data from the Bureau of Meteorology. The update commences at 8:30 pm (AEST) and is done in two stages:

  1. The station and gridded datasets are updated.

    This step usually takes around 3 hours, so after (approximately) 11:30 pm you will be able to access updated data as follows:
    • gridded datasets (all variables)
    • point datasets ordered via the web interface (all variables and formats)
    • point datasets ordered via the web API (only Rainman, monthly and century formats).

  2. The web API's data cache is updated.

    This step usually takes around 5 hours, finishing around 4:30 am the following day. Updated data will then be available for point datasets ordered via the web API (all formats).

    Note: SILO's web API uses a data cache when delivering datasets in standard, alldata, p51, APSIM and CenW formats, as well as customised datasets in CSV and JSON formats.

While you can still request data when SILO is being updated, you should be aware that the datasets may not be fully consistent. For example, a Rainman dataset obtained via the web interface at 2 am will contain updated data for all variables in the Rainman format. However if the same variables are requested via the web API (in one of the formats that uses the data cache, as explained above), you will receive the old dataset because the API's cache is not fully updated until around 4:30 am.

Do you have any metadata describing SILO datasets?

Yes. Metadata are included in the NetCDF gridded datasets, and formal metadata are provided on our metadata page.

How can I import SILO point datasets into Microsoft Excel?

The data must be in a format that Excel can recognise. When requesting point data in:

  • a customised format, you should select the CSV (comma-separated values) option. Excel will be able to read the .csv file directly.
  • one of the standard (i.e. predefined) formats, the data are delivered as plain ASCII text. To import the data into Excel:
    1. In the ‘File Menu’, select 'Open'
    2. In the pop-up window, change the file filter (in the bottom-right corner) from ‘All Excel Files’ to ‘All Files’
    3. Select the SILO data file and click on ‘Open’
    4. In the ‘Text Import Wizard’ pop-up window, select ‘Delimited’ and click on ‘Next’
    5. Select 'Space' in the list of delimiters and click on ‘Next’
    6. Click ‘Finish’ to load the dataset.
    Please note you may need to slightly modify these procedures depending on your version of Excel.

Why can't I enter a location in the drop down box?

When requesting point datasets via the web interface, users can start typing a station number or name in the "Enter location" drop down box. As you type, a list of stations matching what you have entered should dynamically appear. Some users have reported problems when using Internet Explorer. If you experience this problem we suggest you try using another browser such as Google Chrome.

Licence
Creative Commons Attribution 4.0 International (CC BY 4.0) ( http://creativecommons.org/licenses/by/4.0/ )
Last updated
05 Jul 2016