Salve a tutti ragazzi, vi sottopongo una “stranezza” che non riesco ancora a capire. Vi spiego prima cosa faccio.
Per alcune simulazioni atmosferiche scarico automaticamente alcuni dati da un opportuno server. Mi collego a internet e avvio il mio “scriptone” che mi fa tutto automaticamente (simulazione compresa+upload risultati). La funzione che controlla lo scarico dei dati si chiama “g2sub” (vedi più in basso). Fin qui tutto ok.
Ora, siccome la cosa deve avvenire anche in mia assenza ho pensato: perché non verificare (ed eventualmente ripristinare) anche la connessione a internet in automatico?
Così ho creato il seguente script, che fa il controllo ogni 5 minuti e che lancio in background (scusate, l’indentazione viene persa quando incollo qui…):
for (( cycle = 1; cycle <= 10000; cycle++ ))
do
while ! ping -q -w1 www.google.it
do
# se internet non va, ma la chiavetta è connessa
killall pppd
# riconnetti e aspetta
wvdial >& wvdial.log &
sleep 10
done
sleep 300
done
Questo script funziona e navigo tranquillamente. Ma lo script g2sub ora fa le bizze… inizia a scaricare ma poi si blocca. Sul terminale vedo quanto segue:
*
File to download: 17.
Check data available from g2sub Nomads server 1…
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 24636 0 24636 0 0 2298 0 --:–:-- 0:00:10 --:–:-- 3704
GFS Data download from Nomads server 1. File 1/17.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 621k 0 621k 0 0 46672 0 --:–:-- 0:00:13 --:–:-- 60740
GFS Data download from Nomads server 1. File 2/17.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 584k 0 584k 0 0 75 0 --:–:-- 2:11:26 --:–:-- 0
curl: (18) transfer closed with outstanding read data remaining
GFS Data download from Nomads server 1. File 2/17.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 505k 0 505k 0 0 65 0 --:–:-- 2:11:21 --:–:-- 0
curl: (18) transfer closed with outstanding read data remaining …*
Cioè all’inizio tutto bene, poi non riesce a scaricare gli altri file (vi assicuro che nn è un problema del server da cui scarico: quando lo faccio senza lo script di controllo è tutto ok, sono 2 giorni che faccio le prove).
Che ci sia qualche impiccio con i processi in background?
Grazie a tutti e buon lavoro,
Fabio.
PS: in allegato la funzione g2sub (forse non elegantissima, ma testata per mesi e mi funziona perfettamente…).
g2sub()
{
step=1
stop=0
for (( count = 0; count <= stop; count++ ))
do
incr=00
fdate=date +%Y%m%d
checkcurl=1
checkdata1=1
checkday=50
waitnomads=1
STRINGTOSEND=“Stringtosend”
rm -f ${gfsdir}/check*
rm -f ${gfsdir}/run${run}/check*
MICROSUBREG=“subregion=&leftlon=0&rightlon=1&toplat=40&bottomlat=39”
echo
echo “File to download: $nfiles.”
while $checkdata1 -ne 0 -a printf "%02d" $checkday
-ne date +%d -u
] && $waitnomads -ne 5 ]
do
STRINGTOCHECK1=“http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_hd.pl?file=gfs.tprintf "%02d" $run
z.mastergrb2f24&$LEVSTRING$VARSTRING$MICROSUBREG&dir=%2Fgfs.$fdate$run%2Fmaster”
STRINGTOCHECK2=“http://nomad3.ncep.noaa.gov/cgi-bin/g2subset_gfs-0.5.pl?file=gfs.tprintf "%02d" $run
z.pgrb2fprintf "%02d" $incr
&$LEVSTRING$VARSTRING$MICROSUBREG&dir=”
for (( server1 = 1; server1 <= 5; server1++ ))
do
if $checkdata1 -ne 0 -o $checkcurl -ne 0 ]
then
echo
echo "Check data available from g2sub Nomads server 1..."
curl "$STRINGTOCHECK1" -o ${gfsdir}/checkdir1
checkcurl=$?
cd ${gfsdir}
checkdata1=`grep -c -i "NOT PRESENT" checkdir1`
checkgrep=$?
if $checkgrep -eq 2 ]
then
checkdata1=1
fi
sleep 30
else
server1=5
fi
done
if $checkdata1 -ne 0 ]
then
echo
echo "Server 1 problems. Check data update from g2sub Nomads server 2..."
curl "$STRINGTOCHECK2" -o ${gfsdir}/checkdir2
checkcurl=$?
# Check if server 2 data are today data...
cd /root/WPS/util
./g2print.exe -v ${gfsdir}/checkdir2 >& ${gfsdir}/checkgrib2.txt
sed -n '20p' ${gfsdir}/checkgrib2.txt >& ${gfsdir}/checkgrib2row.txt
checkday=`cut -c64 ${gfsdir}/checkgrib2row.txt`
echo
fi
if $checkcurl -ne 0 ] || $checkdata1 -ne 0 -a printf "%02d" $checkday
-ne date +%d -u
]
then
checkdata1=1
checkday=50
waitnomads=$(( $waitnomads + 1 ))
echo “Data not available yet, or network problems. Wait 3 minutes for run $run UTC again, $waitnomads/5…”
sleep 180
fi
done
for (( nfile = 1; nfile <= nfiles; nfile++ ))
do
if $checkdata1 -eq 0 ]
then
echo
echo “GFS Data download from Nomads server 1. File $nfile/$nfiles.”
STRINGTOSEND=“http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_hd.pl?file=gfs.tprintf "%02d" $run
z.mastergrb2fprintf "%02d" $incr
&$LEVSTRING$VARSTRING$SUBREG&dir=%2Fgfs.$fdate$run%2Fmaster”
else
if printf "%02d" $checkday
-eq date +%d -u
]
then
echo
echo “GFS Data (rotating) download from Nomads server 2. File $nfile/$nfiles.”
STRINGTOSEND=“http://nomad3.ncep.noaa.gov/cgi-bin/g2subset_gfs-0.5.pl?file=gfs.tprintf "%02d" $run
z.pgrb2fprintf "%02d" $incr
&$LEVSTRING$VARSTRING$SUBREG&dir=”
fi
fi
# Send the string, get file and check if ok.
curl "$STRINGTOSEND" -o /$rundir/gfs.t`printf "%02d" $run`z.master.grbf`printf "%02d" $incr`
checkcurl=$?
cp /$rundir/gfs.t`printf "%02d" $run`z.master.grbf`printf "%02d" $incr` /$rundir/checklastfile
cd ${gfsdir}/run${run}
checkdata=`grep -c -i "NOT PRESENT" checklastfile`
checkgrep=$?
if $checkgrep -eq 2 ]
then
checkdata=1
fi
# Problem to download single file? Try again that file.
if $checkdata -ne 0 ] || $checkcurl -ne 0 ]
then
if $checkgrep -ne 2 ]
then
sleep 1
nfile=$(( $nfile - 1 ))
incr=$(( $incr - $runstep ))
step=$(( $step + 1 ))
if $step -eq 10 ]
then
stop=1
fi
else
stop=1
fi
else
# Everything seems OK. So keep a copy of data.
cd ${rundir}
/bin/tar -czf $dir_arc/${rundate}_gfs_05.tar.gz gfs.t${run}z.master*
stop=0
fi
wait
incr=$(( $incr + $runstep ))
done
done
Final function check. If not ok, it will be run ftp2m function to try retrieve data.
if $checkdata1 -eq 0 -o printf "%02d" $checkday
-eq date +%d -u
] && $checkcurl -eq 0 -a $checkdata -eq 0 ]
then
echo
echo “Success!!!”
check=0
else
echo
echo “Problems with Nomads G2SUB servers. Try FTP2ME…”
check=1
fi
return $check
}