Tutorial: access to web services
The web services that operate in the background of the Groundwater Data website can be accessed directly from Python. This short tutorial shows you how to use them.
[1]:
import sa_gwdata
The WaterConnectSession object has some useful tools so I suggest loading the background session to use directly:
[2]:
session = sa_gwdata.get_global_session()
I think all the services are documented here.
To pick one service at random. Let’s find a list of wells in the Pike and Katarapko groundwater monitoring networks.
[3]:
response = session.get("GetObswellNetworkData", params={"Network": "KAT_FP,PIKE_FP"})
response
[3]:
<sa_gwdata.waterconnect.Response at 0x1a8918b5d50>
This object provides the raw HTTP response:
[4]:
response.response
[4]:
<Response [200]>
Most responses return JSON (some, the “bulk download” ones, return CSV)
[5]:
response.json[:5]
[5]:
[{'NAME': 'LD 5',
'DHNO': 174806,
'LAT': -34.3242586,
'LON': 140.494952,
'MAPNUM': 692900829.0,
'MAX_DEPTH': 25.4,
'OBSNUMBER': 'KTR043',
'PERMIT_NO': 48454,
'DRILL_DATE': '1999-03-24',
'PURP_DESC': 'OBS',
'AQ_MON': 'Tpl',
'SWL': 18.81,
'TDS': 2909,
'CLASS': 'WW',
'NRM': 'South Australian Murray-Darling Basin',
'LOGDRILL': 'N',
'LITHOLOG': 'Y',
'CHEM': 'N',
'WATER': 'Y',
'SAL': 'Y',
'LATEST_SWL_DATE': '2023-05-24',
'LATEST_SAL_DATE': '2021-07-13',
'OBSNETWORK': 'KAT_FP',
'SWLSTATUS': 'C',
'SALSTATUS': 'N',
'LATEST_OPEN_DEPTH': 25.4,
'LATEST_OPEN_DATE': '2009-06-02'},
{'NAME': 'M27',
'DHNO': 127433,
'LAT': -34.3387926,
'LON': 140.520968,
'MAPNUM': 702900616.0,
'MAX_DEPTH': 183.3,
'OBSNUMBER': 'KTR023',
'PERMIT_NO': 90777,
'DRILL_DATE': '1980-09-25',
'PURP_DESC': 'OBS',
'AQ_MON': 'Ty(conf)',
'SWL': 16.72,
'YIELD': 0.19,
'TDS': 15290,
'STAT_DESC': 'UKN',
'CLASS': 'WW',
'NRM': 'South Australian Murray-Darling Basin',
'LOGDRILL': 'N',
'LITHOLOG': 'N',
'CHEM': 'Y',
'WATER': 'Y',
'SAL': 'Y',
'LATEST_SWL_DATE': '2023-05-23',
'LATEST_SAL_DATE': '1980-09-26',
'LATEST_YIELD_DATE': '1980-09-25',
'OBSNETWORK': 'KAT_FP',
'SWLSTATUS': 'C',
'SALSTATUS': 'N',
'LATEST_OPEN_DEPTH': 180.3,
'LATEST_OPEN_DATE': '1980-09-25'},
{'NAME': 'M30',
'DHNO': 127436,
'LAT': -34.3980486,
'LON': 140.520293,
'MAPNUM': 702900619.0,
'MAX_DEPTH': 72.0,
'OBSNUMBER': 'KTR025',
'PERMIT_NO': 90780,
'DRILL_DATE': '1980-09-11',
'PURP_DESC': 'OBS',
'AQ_MON': 'Ty(conf)',
'SWL': 17.15,
'YIELD': 0.95,
'TDS': 22074,
'STAT_DESC': 'CAP',
'CLASS': 'WW',
'NRM': 'South Australian Murray-Darling Basin',
'LOGDRILL': 'N',
'LITHOLOG': 'N',
'CHEM': 'Y',
'WATER': 'Y',
'SAL': 'Y',
'LATEST_SWL_DATE': '2023-05-24',
'LATEST_SAL_DATE': '1980-09-12',
'LATEST_YIELD_DATE': '1980-09-12',
'OBSNETWORK': 'KAT_FP',
'SWLSTATUS': 'C',
'SALSTATUS': 'N',
'LATEST_OPEN_DEPTH': 72.0,
'LATEST_OPEN_DATE': '2009-06-02'},
{'NAME': 'M31',
'DHNO': 127437,
'LAT': -34.4236476,
'LON': 140.52022,
'MAPNUM': 702900620.0,
'MAX_DEPTH': 30.0,
'OBSNUMBER': 'KTR026',
'PERMIT_NO': 90781,
'DRILL_DATE': '1980-09-16',
'PURP_DESC': 'OBS',
'AQ_MON': 'Qam',
'SWL': 1.28,
'YIELD': 0.0,
'TDS': 12761,
'STAT_DESC': 'CAP',
'CLASS': 'WW',
'NRM': 'South Australian Murray-Darling Basin',
'LOGDRILL': 'N',
'LITHOLOG': 'N',
'CHEM': 'Y',
'WATER': 'Y',
'SAL': 'Y',
'LATEST_SWL_DATE': '2023-05-24',
'LATEST_SAL_DATE': '1981-01-27',
'LATEST_YIELD_DATE': '1980-09-15',
'OBSNETWORK': 'KAT_FP',
'SWLSTATUS': 'C',
'SALSTATUS': 'N',
'LATEST_OPEN_DEPTH': 29.4,
'LATEST_OPEN_DATE': '2009-06-22'},
{'NAME': 'M35',
'DHNO': 127438,
'LAT': -34.4643006,
'LON': 140.535539,
'MAPNUM': 702900621.0,
'MAX_DEPTH': 72.0,
'OBSNUMBER': 'PYP008',
'PERMIT_NO': 90782,
'DRILL_DATE': '1980-06-12',
'PURP_DESC': 'OBS',
'AQ_MON': 'Ty(conf)',
'SWL': 30.16,
'YIELD': 0.25,
'TDS': 3663,
'STAT_DESC': 'CAP',
'CLASS': 'WW',
'NRM': 'South Australian Murray-Darling Basin',
'LOGDRILL': 'N',
'LITHOLOG': 'N',
'CHEM': 'Y',
'WATER': 'Y',
'SAL': 'Y',
'LATEST_SWL_DATE': '2020-06-12',
'LATEST_SAL_DATE': '1985-06-21',
'LATEST_YIELD_DATE': '1980-06-12',
'OBSNETWORK': 'KAT_FP',
'SWLSTATUS': 'H',
'SALSTATUS': 'N',
'LATEST_OPEN_DEPTH': 69.9,
'LATEST_OPEN_DATE': '2009-06-22'}]
The majority of JSON is composed of a relatively flat list, and the sa_gwdata.Response object converts this to a pandas DataFrame:
[6]:
response.df.head()
[6]:
| name | dhno | lat | lon | mapnum | max_depth | obsnumber | permit_no | drill_date | purp_desc | ... | obsnetwork | swlstatus | salstatus | latest_open_depth | latest_open_date | yield | stat_desc | latest_yield_date | pwa | replaceunitnum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | LD 5 | 174806 | -34.324259 | 140.494952 | 692900829.0 | 25.4 | KTR043 | 48454.0 | 1999-03-24 | OBS | ... | KAT_FP | C | N | 25.4 | 2009-06-02 | NaN | NaN | NaN | NaN | NaN |
| 1 | M27 | 127433 | -34.338793 | 140.520968 | 702900616.0 | 183.3 | KTR023 | 90777.0 | 1980-09-25 | OBS | ... | KAT_FP | C | N | 180.3 | 1980-09-25 | 0.19 | UKN | 1980-09-25 | NaN | NaN |
| 2 | M30 | 127436 | -34.398049 | 140.520293 | 702900619.0 | 72.0 | KTR025 | 90780.0 | 1980-09-11 | OBS | ... | KAT_FP | C | N | 72.0 | 2009-06-02 | 0.95 | CAP | 1980-09-12 | NaN | NaN |
| 3 | M31 | 127437 | -34.423648 | 140.520220 | 702900620.0 | 30.0 | KTR026 | 90781.0 | 1980-09-16 | OBS | ... | KAT_FP | C | N | 29.4 | 2009-06-22 | 0.00 | CAP | 1980-09-15 | NaN | NaN |
| 4 | M35 | 127438 | -34.464301 | 140.535539 | 702900621.0 | 72.0 | PYP008 | 90782.0 | 1980-06-12 | OBS | ... | KAT_FP | H | N | 69.9 | 2009-06-22 | 0.25 | CAP | 1980-06-12 | NaN | NaN |
5 rows × 32 columns
[7]:
response.df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 222 entries, 0 to 221
Data columns (total 32 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 name 194 non-null object
1 dhno 222 non-null int64
2 lat 222 non-null float64
3 lon 222 non-null float64
4 mapnum 222 non-null float64
5 max_depth 221 non-null float64
6 obsnumber 222 non-null object
7 permit_no 157 non-null float64
8 drill_date 193 non-null object
9 purp_desc 213 non-null object
10 aq_mon 217 non-null object
11 swl 200 non-null float64
12 tds 184 non-null float64
13 class 222 non-null object
14 nrm 222 non-null object
15 logdrill 222 non-null object
16 litholog 222 non-null object
17 chem 222 non-null object
18 water 222 non-null object
19 sal 222 non-null object
20 latest_swl_date 220 non-null object
21 latest_sal_date 184 non-null object
22 obsnetwork 222 non-null object
23 swlstatus 222 non-null object
24 salstatus 222 non-null object
25 latest_open_depth 213 non-null float64
26 latest_open_date 221 non-null object
27 yield 81 non-null float64
28 stat_desc 57 non-null object
29 latest_yield_date 81 non-null object
30 pwa 101 non-null object
31 replaceunitnum 1 non-null object
dtypes: float64(9), int64(1), object(22)
memory usage: 55.6+ KB
[ ]:
[ ]:
[ ]: