16 de noviembre de 2019

Funding, Infrastructure costs, Images on Amazon

Thanks for your thoughtful consideration of the kinds of nonprofits you support.

For the last three years, iNaturalist has been almost entirely grant funded from individual philanthropists (many with a long history of philanthropy through the California Academy of Sciences), the National Geographic Society, foundations such as the Moore Foundation, and tech companies such as Microsoft through their tAI for Earth initiative, and Google through their tGeo for Good initiative. We also have some project-based funding like the funds we received from the World Wildlife Fund to develop Seek 2.0.

While individual donations from the iNaturalist community currently constitute a small percentage of our overall revenue, we’d love to see that percentage grow. The more sustaining donors we have, the less time we and the development team at CAS have to spend wondering how to meet the next year’s budget requirements.

As a department of the California Academy of Sciences, 15% of additional funds we bring in go toward overhead for CalAcademy, and this includes individual donations. In addition to financial support, they provide office space, legal services, accounting, communications support, and other functions, so the overhead is a way that iNaturalist pays into those costs shared across the institution. The remaining 85% stays within the department to cover staff and operational expenses, like paying Discourse to host this forum, or paying Amazon for image hosting.

The single largest expense for iNaturalist is personnel. iNaturalist has 8 full-time staff in the USA (6 in the Bay Area, 2 remote on the East Coast) and one contractor.

For the CalAcademy fiscal year July 2018 - June 2019, iNaturalist spent $174,000 on infrastructure and miscellaneous expenses. These are the non-personnel related expenses. Like almost everything else about iNaturalist (e.g. users, data, traffic)—except the number of staff— these costs are almost doubling each year.

Donating to iNaturalist 1 (or not) is of course a personal choice. We hope this helps inform your decision. We understand that users have different means and appreciate all of the many ways that people support iNaturalist, financial or otherwise.

Ingresado el 16 de noviembre de 2019 por ahospers ahospers | 0 comentarios | Deja un comentario

15 de noviembre de 2019

Swift libraries remove Restkit, the third-party library

I agree that it’s a bummer when features are pulled, but please understand that I did this for a very good reason. iOS 13 deprecated a number of older developer libraries, and the third-party library we were using to do multi-photo selection was hit hard. It was causing significant amounts of instability, crashes in the app in unpredictable ways, etc. In order to bring multi-photo selection back, I’m either going to have to re-implement that whole library, or find a new third-party library. Both will take time, and are further complicated by the fact that we can’t use Swift libraries in the iOS app right now, since one of the other third-party libraries we’re using (RestKit) won’t compile with Swift. I’ve been working on removing this problematic library for a while now, but it’s slow going because we use it a ton.

So my plan is to keep working to remove Restkit, the third-party library that won’t compile with Swift, so that in the future I’ll be able to choose from the vast menu of Swift libraries that do camera/photo library picker stuff, with lots of support for multi-photo selection.

Ingresado el 15 de noviembre de 2019 por ahospers ahospers | 1 comentarios | Deja un comentario

10 de noviembre de 2019

how many map tiles we generate per minute over

https://forum.inaturalist.org/t/open-test-of-map-tile-improvements/7833

Here’s a chart that shows how many map tiles we generate per minute over the last day (times are in EST). As shown here we’re generating up to about 3000 tiles per minute, or 50 tiles per second. We’re also entering a slower time of year, so during busy months and certainly during events, we render much more than that. For reference, across all iNaturalist and partner websites, the API, and tiles, we process about 15,000 requests per minute (250/sec) right now, which means map tiles represent about 20% of all traffic.

h has me wondering if we need colour to distinguish kingdoms? Why not instead use colour to indicate density from sold blue for single obs to solid red for the highest density, then it will be a heat map similar to this:
ing relatively small squares. in the case of the new test squares, since the squares are relatively large (the tradeoff for clickability and speed of processing), i think a color gradient approach will look bad. below are examples that may help to show what i mean. these examples are using 4px x 4px squares, which are smaller than 8px x 8px squares, but are already clouding up the map. (they use a different color gradient than your example, but they are using iNat data, though probably are scaled a little differently than the test
https://forum.inaturalist.org/t/observation-density/6305/6
(compare the above examples to this: https://forum.inaturalist.org/t/observation-density/6305/6 1, which is a density map from GBIF using iNat data at very fine resolution – very beautiful, in my opinion, but very resource intensive to create and not clickable.)

https://forum.inaturalist.org/t/open-test-of-map-tile-improvements/7833

https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/17

https://forum.inaturalist.org/t/cache-map-data-to-speed-up-loading-and-reduce-demand-on-inat-servers/7767/8

https://forum.inaturalist.org/t/how-do-i-find-topics-that-ive-responded-to/7677/3
Good to know that it existst but a bit hard to find…THis forum software is so weird. BlockquoteYou can also go to your profile and under Activity, click Replies: https://forum.inaturalist.org/u/oneanttofew/activity/replies

You can also go to your profile and under Activity, click Replies: https://forum.inaturalist.org/u/oneanttofew/activity/replies

Ingresado el 10 de noviembre de 2019 por ahospers ahospers | 0 comentarios | Deja un comentario

30 de octubre de 2019

looking-for-inaturalist-observation-map-visualisation-suggestions

https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/11

the api.inat.org and api.gbif.org URLs above won’t actually take you anywhere. but they do provide a GIS tool – in this case, ArcGIS Online – the instructions needed to get map tiles from iNat and GBIF.

you may have noticed that when you load many online maps, they get rendered a section or a tile at a time, not all at once. as you pan beyond the original extent of the map, new sections or tiles get loaded and added to the map. as you zoom in and out, new tiles get loaded to provide a better representation of the map at the given zoom level.

you’ll notice in the tile URLs i provided above, they all contain a {level}, a {col}, and a {row}. ArcGIS will dynamically replace these with a zoom level, an x value (or column index), and a y value (or row index). that tells the API to return a map tile that represents a particular section of the Earth at a given zoom level. when the API provides ArcGIS that tile (usually a picture file – in these cases, all *.png files), ArcGIS puts that tile on the appropriate place in the map, along with all the other tiles needed to fully display the map.

here’s how you can create a map with one of these tilesets in ArcGIS Online:

go to https://www.arcgis.com 1
at the top of the screen, find Maps, and click on it. (you don’t have to sign in.)
that should get you to a screen with a blank map. at the top right of that screen, click Modify Map.
that wll add some editing options to the toolbar below. find Add in that tool bar (on the left side of the screen), and click on that, then select “Add Layer from Web” from the drop list.
that will give you a pop up. in that pop-up, select “A Tile Layer”, then fill in the URL (using one of the URLs i provided before), a title for it (which is what show up in the left Contents pane in the screenshots), and credits (which is what shows up in the bottom right corner of the map in the screenshots). then click Add Layer.
finally, you may want to use a basemap that is less busy than the default topo basemap. you can either select one by clicking the Basemap button in the map toolbar, or searching for one via Add > Search for Layer. (ArcGIS has lots of layers available.)
if you don’t like ArcGIS, you can also do something similar in other GIS tools. the specific steps will differ, of course, but the concept is the same. (the URLs may differ in the other tools, too. for example, QGIS and Leaflet.js expect {z}, {x}, and {y} in the URL instead of {level}, {col}, and {row}.)

hope that helps. if you have other questions, feel free to ask.

also note there are a few more advanced things that can be done. for example, you can use different tilesets for different zoom level ranges. so for example, the hexagons that GBIF provides may look okay at that size in my example at a continent level, but they might look too much like points at a higher (more granular zoom level). so you might choose to use relatively bigger hexagons at higher zoom levels by changing the hexPerTile value in the URL (smaller values result in bigger hexagons in this case). you can also modify the transparency of the tiles within your particular GIS tool.

2

Reply

pisum
1
8d
it occurred to me last night / this morning that i could probably take the results from the UTFGrid tiles that the iNaturalist API serves up and use them to create my own custom observation maps. so when i got some time this evening, i tried to create my own custom map based on UTFGrid data, and it actually worked out better than i expected. i think UTFGrids might be a little less precise, but they are much more customizable than either iNat’s standard observation tiles or GBIF’s tiles, and they offer all the query options that iNat provides. below is an example of all observations from iNat. i compared it against another map i created from GBIF (https://forum.inaturalist.org/t/observation-density/6305/6), and it looks similar (which is good from a data quality perspective). i’ll probably work on it a little bit more when i get time just for fun, but if anyone is interested in it, i’ll clean it up a little bit more and can share the code when it’s more functional.

tbd%20utfgrid%20alternative%20implementation
tbd utfgrid alternative implementation.jpg
1534×1330 133 KB
UPDATE: i was doing some more sanity checks on the data before i got too far down the road of coding something that turned out later to be flawed – so far, things look ok – and i just wanted to share some more maps that i thought were interesting.

here’s a map that shows observations over the last 4 years (r=2019, y=2018, g=2017, b=2016, brightest dots represent at least 20k observations). you can sort of see how iNat usage is spreading across the world:
tbd%20obs%20by%20year%20r2019%20y2018%20g2017%20b2016%20max20000
tbd obs by year r2019 y2018 g2017 b2016 max20000.jpg
1535×790 102 KB
this map shows observations over the last 4 months (r=Oct, y=Sept, g=Aug, b=July, brightest dots represent at least 2500 observations):
tbd%20obs%20by%202019%20month%20r10%20y9%20g8%20b7%20max2500
tbd obs by 2019 month r10 y9 g8 b7 max2500.jpg
1536×785 120 KB
this map shows observations Monarchs over the last 4 months (r=Oct, y=Sept, g=Aug, b=July, brightest dots represent at least 25 observations). you can sort of see a shift in observations over these months related to migration:

https://forum.inaturalist.org/t/looking-for-inaturalist-observation-map-visualisation-suggestions/7322/11

k… i’ve been coding a bit, and i’m at a point of diminishing returns for further coding, i think. i didn’t get to the point of producing a mapping interface, but the code is here (https://github.com/jumear/stirfry/blob/gh-pages/iNat_UTFgrid_based_custom_density_map.html 2), and hopefully it’s relatively easy to understand and tinker with. you’re welcome to adapt it as you please.

here are some examples of different custom maps i created using the UTFgrids:
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=1 2
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=2 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=3 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=4 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=5 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=6 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=7 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=8 1
https://jumear.github.io/stirfry/iNat_UTFgrid_based_custom_density_map.html?example=9 1

UPDATE:

one last contribution here: https://jumear.github.io/stirfry/iNat_UTFgrid_based_density_map_for_Leaflet.html
(code: https://github.com/jumear/stirfry/blob/gh-pages/iNat_UTFgrid_based_density_map_for_Leaflet.html)

the previous examples were built in a static map viewer that i cobbled together. but this new example is built as an extension of Leaflet.js. so it might be easier to tinker with for those who are familiar with Leaflet.js, and this example is also easier for exploration since you can pan and zoom. the markers are also created a little differently here, in a way that is less resource intensive but is a little less flexible (tradeoffs).

that’s it for now.

hmm… if you’re trying to do heatmaps, then you should try the tilesets served up by iNat 1 and GBIF 1, if you haven’t already.

iNaturalist’s tiles are a little bit uglier in most cases (in my opinion), but they can be set to any color using the color parameter. GBIF’s tiles look a little better (in my opinion), and the polygon tilesets in particular offer lots of options for formatting, though there are fewer color options available.

here are 4 quick examples of observation tilesets for Diloma concameratum (Wavy Top) that i pulled up in ArcGIS Online over a dark basemap (each snapshot is preceded by the tileset URL i used for that layer):

https://api.gbif.org/v2/map/occurrence/density/{level}/{col}/{row}@1x.png?srs=EPSG:3857&style=orange.marker&taxonKey=5797922 2
tbd%20heatmap%20gbif%20marker
tbd heatmap gbif marker.jpg
1236×957 117 KB
https://api.gbif.org/v2/map/occurrence/density/{level}/{col}/{row}@1x.png?srs=EPSG:3857&style=iNaturalist.poly&taxonKey=5797922&bin=hex&hexPerTile=16 2
tbd%20heat%20gbif%20hex%2016
tbd heat gbif hex 16.jpg
1236×957 120 KB
https://api.inaturalist.org/v1/heatmap/{level}/{col}/{row}.png?color=orange&taxon_id=861925 1
tbd%20heatmap%20inat%20heat
tbd heatmap inat heat.jpg
1236×957 111 KB
https://api.inaturalist.org/v1/colored_heatmap/{level}/{col}/{row}.png?color=purple&taxon_id=861925

Ingresado el 30 de octubre de 2019 por ahospers ahospers | 2 comentarios | Deja un comentario

23 de agosto de 2019

Analysing iNat data

While I haven’t done it with iNaturalist data, if you are trying to start doing data analysis on large datasets I suggest learning some python. Packages in python such as Numpy, Pandas, Scipy, Scikit-learn and Matplotlib go awfully far for data analysis.

I suspect it is possible to use pyinaturalist 3 and the API reference 1 for getting data out of iNaturalist as well, but it seems like you were already able to extract to excel already. https://pypi.org/project/pyinaturalist/ https://www.inaturalist.org/pages/api+reference Data analysis and visualization is really a field of its own. There are many different software applications, like R, python, etc, but you really need to have a working knowledge of these programming languages. If you’re only familiar with Excel, that might be the best place to start. There are not any programs specifically for easily working with iNat data.

https://forum.inaturalist.org/t/analysing-inat-data/6001/3

https://pypi.org/project/pyinaturalist/

https://www.inaturalist.org/pages/api+reference

?interpolate_coordinates=true

Find observations missing a location
https://www.inaturalist.org/observations?quality_grade=any&place_id=any&not_in_place=91708,97391&user_id=ahospers&verifiable=any works!

But there is still a huge need for classical projects: i.e. projects to which observations are manually added, or added by virtue of observation fields that are required. To my mind these are "tagged" projects. (in iNat logic one traditionally "added observations to projects", but for similar projects on iSpot we added tags and created projects to filter on the tags - the net effect is the same, a lookup table of project-tag and observation, that can be used as a filter - either alone or with other filters).
The addition of a project-tag filter to the new projects will make the "traditional" projects into "collection type" projects too, while preserving the Project Icons on the observations.
((the additional functionality of defining observation fields from within a project is an extremely useful feature.))

A nitpick: is the central filter a Location filter (using google places) or a Place filter (using user-created iNat polygons) or both? It would help if the terminology was standardized with the observations pages and its filters.

?interpolate_coordinates=true

photos/?advanced=on

Here's the link for creating a traditional project: https://www.inaturalist.org/projects/new_traditional

Ingresado el 23 de agosto de 2019 por ahospers ahospers | 0 comentarios | Deja un comentario

17 de agosto de 2019

KNNV Drielanden Lepidoptera night (Trip)

KNNV Nachtvlinder nacht

https://www.inaturalist.org/identifications?user_id=juliereid 3

Other parameters you can add:

&taxon_id=1234
&category=maverick|supporting|improving|leading Default: any
&current=false|any Default: true
&for=self|others|any Default: any
?interpolate_coordinates=true

Find observations missing a location
https://www.inaturalist.org/observations?quality_grade=any&place_id=any&not_in_place=91708,97391&user_id=ahospers&verifiable=any works!

https://www.inaturalist.org/identifications?user_id=juliereid&for=others
You can play with the other parameters, as well – adding different &PARAMETER=SETTING options. Choose between options separated by the “|”. It’s safe; you shouldn’t be able to break anything, just get

But there is still a huge need for classical projects: i.e. projects to which observations are manually added, or added by virtue of observation fields that are required. To my mind these are "tagged" projects. (in iNat logic one traditionally "added observations to projects", but for similar projects on iSpot we added tags and created projects to filter on the tags - the net effect is the same, a lookup table of project-tag and observation, that can be used as a filter - either alone or with other filters).
The addition of a project-tag filter to the new projects will make the "traditional" projects into "collection type" projects too, while preserving the Project Icons on the observations.
((the additional functionality of defining observation fields from within a project is an extremely useful feature.))

A nitpick: is the central filter a Location filter (using google places) or a Place filter (using user-created iNat polygons) or both? It would help if the terminology was standardized with the observations pages and its filters.

Ingresado el 17 de agosto de 2019 por ahospers ahospers | 2 comentarios | Deja un comentario

KNNV Drielanden Lepidoptera night (Trip)

KNNV Nachtvlinder nacht

Ingresado el 17 de agosto de 2019 por ahospers ahospers | 0 comentarios | Deja un comentario

11 de agosto de 2019

20190811 Drielanden KNNV Groningen InventarisatieProject (Trip)

20190811 Drielanden KNNV Groningen InventarisatieProject Ilse, Hugo, Thorhold, Andre, Wil, Stella

Ingresado el 11 de agosto de 2019 por ahospers ahospers | 26 observaciones | 0 comentarios | Deja un comentario

10 de agosto de 2019

Find/Exclude observations without a location?

Find/Exclude observations without a location?
https://forum.inaturalist.org/t/find-exclude-observations-without-a-location/3571
https://forum.inaturalist.org/t/how-to-use-inaturalists-search-urls-wiki/63
, weird twist on place exclusion: While searching through casual, I know how to exclude humans, pets, potted plants. and records without photos. If I also want to exclude ones without any location given, what is the proper exclusion term? These might not work: place_id=null or not_in_place=true (heh)

Then, I might as well also ask what the term is for excluding ones with no observation date. Thanks for advice!

Hm reading back over my post, it’s confusingly written. To be clear I want to search for observations that definitively do have a location, and exclude the ones without a location. These might not work for that: place_id=any or not_in_place=false

But since others may even specifically want to search for records without place or without date to help people get those particular orphaned observations in shape, I’ll leave the prior unclear post as an implied question about that too.
Find/Exclude observations without a location?
https://forum.inaturalist.org/t/find-exclude-observations-without-a-location/3571
https://forum.inaturalist.org/t/how-to-use-inaturalists-search-urls-wiki/63

To your question about location vs. place, the version released next week will say "Places" instead of "Locations" for clarity (and other changes needed to be consistent with the explanation text—these were the designs used to create the feel and core functions, but not the final copy).

?interpolate_coordinates=true

photos/?advanced=on

Here's the link for creating a traditional project: https://www.inaturalist.org/projects/new_traditional

Ingresado el 10 de agosto de 2019 por ahospers ahospers | 0 comentarios | Deja un comentario

09 de agosto de 2019

BioCaching with iNaturalist Trips

https://apps.apple.com/us/app/biocaching/id827737251

BioCaching: Rediscover Nature
https://forum.inaturalist.org/t/trips-feature-on-inat/3061/12
BioCaching puts the power to conduct a biodiversity resurvey in your hands. Search for and revisit places where plants and animals have been observed in your community, or anywhere in the world. By using this app you will update our knowledge of current biodiversity distributions.

Our app works together with iNaturalist (an online platform for recording and sharing biodiversity observations). Learn more about the iNaturalist community and sign-up to the platform at http://www.inaturalist.org (you can also sign-up through the BioCaching app if you're not already a member).

With one click, you will see all of the plants and animals that have been observed around you. Next, choose previously observed species that you can recognize and would like to try and resurvey. You are now ready to conduct a ‘trip’, attempting to re-document those species. Start looking and record whether or not you are able to relocate each frog, toad, tree, bee, spider, shrub or whatever comes up on your list. If you find what you are looking for, record new field observations. Your entire trip and your finds will be recorded on iNaturalist.org and all verified reports will become new records available to scientists. The BioCaching app is a concrete way for you to connect with nature and bear witness to habitat loss and biodiversity change.

-----

The BioCaching app, was developed and designed by Andy Jeffrey in partnership with the California Academy of Sciences Library, Academy Citizen Science and iNaturalist and was funded by the Institute of Museum and Library Studies National Leadership Grant, Connecting Content: A Collaboration to Link Field Notes to Specimens and Published Literature, LG- 05-010-0048.

https://apps.apple.com/us/app/biocaching/id827737251

https://forum.inaturalist.org/t/trips-feature-on-inat/3061/12

Ingresado el 09 de agosto de 2019 por ahospers ahospers | 3 comentarios | Deja un comentario