Some files/years seem to be missing from CMIP5 data in both the DRSv2 three and the paths returned by search_replica.py when searching for alternative file locations. The issues I ran into are:
1. CMCC-CESM , rcp85 , monthly , variable: tas , r1i1p1 :: data for the years/months 209601 to 210012 are missing
2. HadGEM2-CC , rcp85 , daily , variable: tasmax , r1i1p1 :: no data for 20051201 to 20991231
3. HadGEM2-ES , rcp85 , daily , variable: hfls , r1i1p1 :: no data for 20951201 to 21001231
Would it please be possible to add these missing data?
While I'm updating the database now, I won't make it copy it to the official version until I finish other downloads I'm doing, so it won't be there until Monday. I think the DRSv2 gets updated in the weekend too
It seems I ran again in the issue with some HadGEM2-CC data not being available (rcp85 , daily , variable: tasmax , r1i1p1, v20120531), there are no data for 20051201 to 20991231 in
My script suggest I was accessing those files in Dec 2016, so it seem they may have got lost since then?
ceda changed their server so the data downloaded in December is in
if you use the search_replica.py script with the -a flag it will return all available versions not just the latest, so if there are two options for the latest you'll see both of them. DRSv2 will have to make a choice between latests too and sometimes is one which is broken. The dataset should be cleaned up for this to be avoided but it's not clear whose responsibility it is. In this case I'll delete this so DRSv2 won't pick it up anymore at its next update during the weekend.
I'm fixing the same issue for the variables: ta, tasmin, rsds, psl, rhs, ua, va, zg, hus and hur. I've deleted the incomplete versions from unofficial and from the db and I'm downloading the full ensembles from the new ceda server (download is very slow)
Thank you Paola!
Thanks for your help with downloading the missing files.
re 1) But daily tasmax data from CMCC-CESM is available until including year 2100, see here /g/data1/ua6/DRSv2/CMIP5/CMCC-CESM/rcp85/day/atmos/r1i1p1/tasmax/v20120730
so I would expect that also monthly tas should be available for these years...?
All the available files have been downloaded , unfortunately although the daily files are available until year 2100, the monthly files stop consistently for all variables at 2095 included.
Daily tas FGOALS-s2 r1i1p1 historical has two missing years: 1973, 1974.
There's no available version of this online, not even among replicas, so this cannot currently be fixed
There are missing years in this file:
When I use the cdo -showyear command, the output only reveals these years:
[lgv561@raijin5 r12i1p1]$ cdo showyear tnxETCCDI_mon_EC-EARTH_historical_r12i1p1_185001-201212.nc
1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1850
All the other files in this model's realisation are fine.
Received and problem confirmed. Lawson Hanson will update this dataset to the newest version.
Internal reference: RT#94760
Sorry for not being more thorough earlier, but I found these files are missing years as well.
I have not searched through every index/model just those I need.
Appologies, this one too
From Lawson Hanson:
I just noticed that the file referenced as:
was probably meant to be:
i.e., with "_mon_" between "tnnETCCDI" and "CSIRO-Mk3-6-0"; and I have checked that (it had 144 years) and have re-downloaded a replacement file which has 156 years.
Thank you Lawson
Hi appologies for opening this up again but these files also have the same issue:
I have re-downloaded the three files specified, checked them; they appear to be OK now.
Best regards, Lawson
This version of the Canada Extreme Indices data scheduled to be replaced with the newer version (currently in ua6/processed/climdex) on 23/3/2017.
Hi Claire and Lawson,
I've found missing CMIP5 'tos' files for the CSIRO-Mk3L-1-2 model:
Could you please look into these
this is not maintained by any of us, these are links that Nadja created previously, I believe she cannot maintain them herself either. There should be a copy in /g/data1/w35/ which might have been updated more recently. Also currently ua6 which is the project the links would refer to is being transferred to a new disk by NCI and we cannot make any changes to this collection while this is happening (Claire Carouge sent an e-mail the other day about this).
Since soon the same transfer will happen for ua8 and I suggest you use the /g/data1/w35/CMIP5_Archive or try one of the other available tools as an alternative.
We'll have to delete the CMIP5_Archive from ua8 in the next few months, it will stay in w35.
Finally this are files from the CSIRO Mk model so they should be officially published by NCI (as it's the ACCESS model). Any model published by NCI is available under /g/data1/ua6/authoritative.
There's a CMIP5 induction on the NCI training site training.nci.org.au that has all this information