News on GIS, Remote Sensing, R, Web-mapping, Leaflet, Data Visiualization & - Management. Tutorials on Leaflet R raster, rgdal and other Open Source. Information on GIS, Mapping, Remote Sensing, Geodata, Geospatial news
I’d like to draw your attention to the GEO Data Technology Workshop which will take place from 23-25 April 2019 in Vienna, Austria. The workshop will focus on exploring how to best make available Earth observations-based knowledge for international policy frameworks, including the United Nations 2030 Agenda for Sustainable Development, the Sendai Framework on Disaster Risk Reduction and the Paris Climate Agreement.
With the EGU General Assembly taking place just two weeks earlier on 7–12 April 2019, Vienna will be the epicenter for geoscientists in April. The EGU is the second largest conference in Vienna, covering all disciplines of the Earth, planetary, and space sciences.
Finally, PyDays Vienna will be held again this year on 3-4 May in cooperation with Linuxwochen Österreich. Within just two years, this conference became Austria’s largest Python event, covering a broad range of topics from data science to web development.
More information on the three conferences can be found at:
A GeoJSON file (Simple Feature Access standard) describing the region of interest is needed. The easiest way to create a ROI polygon is by converting an existing Layer into a bounding box in QGIS (Vector / Research Tools / Extract Layer Extent) or by creating a new layer and manually mapping the ROI polygon:
Layer / Create Layer / New Shapefile Layer
Edit / Add Feature
Either way, results can be exported as GeoJSON via Export / Save Features As / GeoJSON. Keep in mind to choose EPSG:4326 (WGS84) as CRS when exporting.
Get SENTINEL-1 data
Download through Open Access Hub
Downloading a bulk of Sentinel images, either S1 or S2, is not very smooth if you use the GUI from the Open Access Hub. However, the sentinelsat Python package does provide a nice interface to the API:
from sentinelsat import SentinelAPI, read_geojson, geojson_to_wkt
api = SentinelAPI('username', 'password')
footprint = geojson_to_wkt(read_geojson('/path/to/roi.geojson'))
products = api.query(footprint,
beginposition='[YYYY-MM-DDT00:00:00.000Z TO YYYY-MM-DDT00:00:00.000Z]',
StaMPS is compatible with the output generated by the ESA SentiNel Application Platform (SNAP) after the version 6.0. SNAP allows the user to define a series of xml files which contain user defined processing workflow by using its Graph Builder. These files can be used to run SNAP processing in batch mode by using the GPT command (Graph Processing Tool).
snap2stamps contains a set of graphs, together with python wrappers that allow you to automatise the interferogram processing chain for single master interferograms compatible with StaMPS PSI. Information about the provided functionalities and their instructions can be found in the user manual provided within snap2stamps, which already plans newer releases increasing functionality and compatibility maintenance of SNAP-StaMPS chain. A reference to the software package (Foumelis et al., 2018) can be found in the manual.
The following steps are needed and provided by snap2stamps:
Sort the slave images in separated folders corresponding to their acquisition time.
Slave splitting (and slice assembling if needed) and annotated orbits replacement with Precise/Restituted Orbits.
Coregistration and interferogram generation done per slave and subswath. Here auxiliary files as DEMs and subset using a bounding box is performed. Both amplitude coregistered master-slave stack and interferogram with elevation and orthorectified latitude and longitude coordinates are saved as output.
StaMPS export is done providing for each pair coregistered stack and interferogram generated StaMPS compatible products.
snap2stamps contains the following files:
project.conf – file with parameters and paths needed for the processing
slaves_prep.py – script for sorting slaves into the expected folder structure
splitting_slaves_logging.py – script for slave splitting (and assembling if needed) and orbit correction
coreg_ifg_topsar.py – script for master-slave coregistration and interferometric generation
stamps_export.py – script for ouput data generation in StaMPS compatible format for PSI processing
Note that snap2stamps requires Python 2.7.
Select optimal master in SNAP using Radar / Interferometric / InSAR Stack Overview
Perform subsetting of whole image using TOPSAR Split via Radar / Sentinel-1 TOPS / S-1 TOPS Split. Set the processing parameters
subswath (IW 1-3),
polarization (vertical sent, vertical returned – VV), and
bursts (using the bursts-slider) in the respective tab.
Get LAT/LON MIN/MAX (bounding box) for PSI area of interest. This can be obtained e.g. from ROI polygon in QGIS (Layer Properties / Metadata / Properties / Extent).
(To keep a backup, you might want to simply copy your snap2stamps bin and graphs folders to your working directory). Edit project.conf to set up the configuration for your project. An example configuration file might look like this:
Move the master (zip + TOPS – Split Output) to the directory master in your PROJECTFOLDER.
Make sure that all slave images (zip) are stored in the subfolder slaves in the PROJECTFOLDER.
Check if all libraries are available for your Python 2 installation (you might need to pip install pathlib). I strongly recommended to use a corresponding conda environment for Python 2.7: source activate snap2stamps.
Run the python scripts of snap2stamp directly in your shell:
# slave sorting
python slaves_prep.py project.conf
# slave splitting and orbit correction
# (approx. 50 seconds per slave)
python splitting_slaves.py project.conf
# master-slave coregistration and interferometric generation
# (approx. 180 seconds per slave)
python coreg_ifg_topsar.py project.conf
# ouput data generation in StaMPS compatible format
# (approx. 30 seconds)
python stamps_export.py project.conf
The data final output structure after performing the last step should contain these four folders: rslc, diff0, geo and dem within your INSAR_master_date folder.
Check for empty interferograms. If any exist, remove files containing the date of the empty file from the folders rslc and diff0. Otherwise, this will throw warnings related to 0 mean amplitude during the final preparation step use in stamps (i.e. mt_prep_snap) and eventually screw up the selection of PS candidates.
Prepare the data set for final use in StaMPS using mt_prep_snapwith <yyyymmdd> being the master date. The path after the date is the full filepath to the INSAR_master_date folder:
mt_prep_snap /home/username/…/INSAR_master_date/ DA
The amplitude dispersion index is a value that describes the amplitude stability, which is used to preselect pixels and therefore reduces the number of pixels for the phase analysis. It is calculated as , where is the standard deviation and is the mean of a series of amplitude values.
The recommended range for is 0.4-0.42. The higher the threshold, more pixels will be selected for phase analysis. Note that surfaces like water and vegetation, where amplitude is instable, exhibit higher values than bedrock outcroppings or man-made strucutres (i.e. most likeley PS pixels). See Ferretti et al. (2001) and Hooper et al. (2007) for further information. Additional arguments for mt_prep_snap are only needed for large areas (i.e. if multiple patches are processed).
Finally, launch matlab to continue with StaMPS PS analysis.
This is the basic processing chain for obtaining Sentinel-1 SLC data and preparing them for further use in StaMPS.
I am currently involved in several projects that rely on Persistent Scatterer Interferometry (PSI), a radar-based technique that belongs to the group of differential interferometric Synthetic Aperture Radar (SAR).
Specifically, I am using StaMPS/MTI to analyse Sentinel-1 SLC data. StaMPS/MTI written in Matlab and C++. Currently, this seems to be the most cost-efficient software to perform PSI software, since commercial software (e.g. GAMMA or SARScape) is quite expensive. Even though a Matlab-license is required, StaMPS code is more or less open source (that is, apart from built-in Matlab-functions).
Having mainly worked with optical data and R / Python / QGIS before, it took me some time to get everything running as intended, especially since the documentation is rather rudimentary, at least if one is starting to explore this topic. I therefore decided to document all relevant steps in a series of posts, starting with the very basics: installation and setup instructions.
This documentation is intended to provide guidance on how to setup StaMPS for the first time and is based on the excellent SNAP-StaMPS Workflow compiled by Thorsten Höser as well as discussions from MAINSAR, the step forum on snap-stamps workflow and the stamps-tool as well as Ask Ubuntu. Acknowledgements and thanks to all people contributing to these discussions.
Please note that this summary has been written and tested on a certain machine using a specific user. Even though I tried to generalize code as much as possible, certain paths may have to be adjusted accordingly.
Theoretical Background on PSI
Guidance given in this summary is purely technical. However, it is absolutely necessary to know the basics of the theoretical background of radar remote sensing and InSAR, otherwise the whole adventure of StaMPS PSI analysis is doomed to fail, since you will be lost given the large number of parameter settings. Based on Thorstens’ recommendations, I have compiled a list of potentially helpful sources:
Install a Linux distribution of your choice. Since I am a happy Ubuntu user since quite some time, this instruction refers to Ubuntu 18.04 LTS (with vanilla GNOME). Commands below should work on any Debian-based distro, though. If you are using another distribution, some commands have to be adjusted accordingly.
Install MATLAB for Linux. Things may go haywire here. Note that this may cause some trouble with respect to version incompatibilities of system libraries and libraries contained in the MALAB-installation (e.g. libstdc++6, gcc-4.9). Possible fixes may be found e.g. on the MathWorks forum or AskUbuntu.
Install dependencies – these include GNU awk, the tcsh unix shell, the matlab support package (Ubuntu-specific), the build-management-tool, and the C++ compiler:
If you intend to include tropospheric correction methods into the InSAR processing chain, install the Toolbox for Reducing Atmospheric InSAR Noise (TRAIN). This is optional, but will increase the accuracy of your results. TRAIN can be used in StaMPS (step 8) to remove tropospheric delays. Note that TRAIN does have additional dependencies (see manual for full description), such as e.g.:
These are the basic installation steps and software requirements. Let’s move on to editing some config-files.
Edit the StaMPS config file (e.g. via vim ./StaMPS/StaMPS_CONFIG.bash – or wherever your StaMPS installation is located) to contain the correct paths to StAMPS, snap2stamps, SNAPHU. Note that this is jus a generic example, you have to adjust the paths accordingly:
# set environment variables:
#if triangle and snaphu are not installed through the repositories (i.e. compiled locally):
export MATLABPATH=$STAMPS/matlab:`echo $MATLABPATH`
# use points not commas for decimals, and give dates in US english
source the StaMPS config file upon opening terminal by adding it to your .bashrc:
On 3 December, Orbital Reflector (ORS-1) was launched aboard a Falcon-9 rocket, and will continue to orbit Earth for about two months. Orbital reflector is a temporary satellite which does neither serve any scientific nor technical purpose. Rather, the installation designed by Trevor Paglen and the Nevada Museum of Art is supposed to stimulate contemplation:
“As the twenty-first century unfolds and gives rise to unsettled global tensions, Orbital Reflector encourages all of us to look up at the night sky with a renewed sense of wonder, to consider our place in the universe, and to reimagine how we live together on this planet.”
The launch of such nonfunctional satellites is not without controversy (see e.g. the criticism of Humanity Star), especially among astronomers and astrophotographers. According to critics, there are several other bright satellites in Earth’s orbit that are visible to the naked eye while actually serving a useful function (most notably ISS) instead of being mere space garbage.
However, I shall leave it to the kind reader (particularly the stargazers among them) to conceive an opinion on this topic.
Photo Credit: James Lee: 91 Minutes of the Night Sky , CC BY 2.0
I am currently attending the EO4Alps conference in Innsbruck, Austria. The objectives of this interesting workshop are:
Assessing the status and priority interests in the European Alpine Region for Earth Science, geo-information and operational monitoring;
Reviewing the main activities, projects and initiatives taking place in the different Alpine application areas, where Earth observation from space can contribute;
Evaluating the potential of the increasing observation capacity by satellites to address needs for science, applications and information services in the European Alpine region;
Characterising and addressing data collection and analysis gaps and define high-level requirements for supporting ICT solutions to advance EO data infrastructure capabilities in the Alpine region.
The diverse programme features seven different sessions, covering various topics such as atmospheric processes, natural hazards, environmental and climate change, natural resources, economic development and ecological networks, combination of ground-based and satellite systems as well as infrastructure and data analysis.
I’d like to point out a study conducted by colleagues from the Austrian Institute of Technology, the University of Natural Resources and Life Sciences Vienna and the University of Salzburg. In this study, we have analysed indicators for the sustainable development of cities that have the greatest potential for their underlying data to be measured by means of remote sensing.
I have received multiple requests over the last few months to provide information on how I created the combined return level plots when using both annual maxima series and partial duration series. I have put off the evil hour to provide these scripts since I intended to come up with a nice ggplot solution.
As luck would have it I just needed to create another one of these combined return level plots for a conference poster, and I really shuddered at the thought of digging up my old R scripts I had used for that occasion. Even though it immediately turned out that my fear was not clearly not unfounded, for lack of time I have not been able to rework this behemoth of code (which is based on plot.fevd() from the extRemes package)to make it as tidy as initially intended. However, I think I have at least been able to restructure the basic plotting function rlplot() in a way that it might be understandable to other users. I might completely rework this function by building it from scratch one day, since the current version is an extremely verbose and somewhat very hacky solution. But it works, and that should be sufficient for now.
Since the function is somewhat cumbersome, I have provided it via our GitLab repository.
Attached is a short example of application, based on the same precipitation data set I have already used for the other posts on this topic. The functions required to run the code are read_ehyd() and rlplot().
# load required packages
# get data from eHYD