Arquivos de periódicos de fevereiro 2019

04 de fevereiro de 2019

Python upload script

Since pyinaturalist recently came out I thought it would be a good time to try and write a script to automatically upload files to iNaturalist. Few if any example scripts are out there and this should make it easier for other people to write one which matches their workflow.

This script assumes a large number of photos of the same species. This might happen for example if you were trying to map every tree on a property. The workflow would consist of taking a single geotagged photo of each individual then separating out the photos so each one is in a folder which starts with its taxon ID. For example aphids would go in a folder named '52381' or '52381 Aphids' or '52381-Aphididae'

If you don't have python, I suggest installing Anaconda then pyinaturalist. You will then need to get an app ID.

Copy this script, paste it to a text file renamed to end in .py, add your user name, password, app id, secret, and the time zone of the photos. Then run the script. It should upload everything jpg file in the folder as the file you select.


# Input your user name here:

user = ''

# Input your password here:

passw = ''



# Input your app ID and secret here:

app = ''

secret = ''

# Input the time zone for the photos here, options can be found at the

# website below

# https://gist.github.com/mjrulesamrat/0c1f7de951d3c508fb3a20b4b0b33a98

time_zone = 'America/Los_Angeles'

# tkinter used to choose a file

from tkinter import filedialog

from tkinter import Tk

# os used to get a folder name

import os

# pillow used to get exif data from the photos

import PIL

from PIL import ExifTags

# This is used to upload the photos.

import pyinaturalist

from pyinaturalist.rest_api import create_observations

from pyinaturalist.rest_api import get_access_token

print("Running")

# This code lets you choose a photo, can delete and replace with folder_name=''

root = Tk()

filename =  filedialog.askopenfilename(initialdir = "/",

                                    title = "Select one of the .jpg files in "

                                    "the folder to be uploaded. All files in "

                                    "the folder will be uploaded. The folder "

                                    "name should start with the taxon number",

                                    filetypes = (("jpeg files","*.jpg"),

                                    ("all files","*.*")))

root.withdraw()

folder_name = os.path.dirname(filename) +'/'

print('Uploading all photos in ' + folder_name + ' as a unique observation')

# Makes a list of all files in the folder inside element 2 of a tuple

for file in os.walk(folder_name):

    if file[0] == folder_name:

        files = file

# Creates list of all the file paths for every file in the folder.

file_paths = []

for file in files[2]:   # All files are in files[2]

    file_path = files[0] + file  # files[0] has the path to the folder

    file_paths.append(file_path) # Makes a big list of paths

# This function returns the latitude and longitude of a .jpg image

def get_lat_long(image):

    # Gets all the exif data from the photo

    exif = {

        PIL.ExifTags.TAGS[k]: v

        for k, v in image._getexif().items()

        if k in PIL.ExifTags.TAGS

    }

    # From all the exif data, pulls the GPS data

    gps_info = exif.get('GPSInfo')

    # The GPS data is in a odd format, so have to dig for it a bit. This was

    # only tested on files lightroom tagged.

    latitude_direction = str(gps_info.get(1)[0])

    latitude_degrees = float(gps_info.get(2)[0][0])

    minutes = float(gps_info.get(2)[1][0])

    multiplier = float(gps_info.get(2)[1][1])

    latitude_minutes = minutes/multiplier

    seconds = float(gps_info.get(2)[2][0])

    multiplier = float(gps_info.get(2)[2][1])

    latitude_seconds = seconds/multiplier

   

   

    # The sign is changed depending on if this is N or S

    if latitude_direction == 'N' or latitude_direction == 'n':

        latitude = latitude_degrees+latitude_minutes/60 + latitude_seconds/3600

    elif latitude_direction == 'S' or latitude_direction == 's':

        latitude = -(latitude_degrees+latitude_minutes/60 + latitude_seconds/3600)

       

    longitude_direction = gps_info.get(3)[0]

    longitude_degrees = gps_info.get(4)[0][0]

    minutes = float(gps_info.get(4)[1][0])

    multiplier = float(gps_info.get(4)[1][1])

    longitude_minutes = minutes/multiplier

    seconds = float(gps_info.get(4)[2][0])

    multiplier = float(gps_info.get(4)[2][1])

    longitude_seconds = seconds/multiplier

    # The sign is changed depending on if this is E or W

    if longitude_direction == 'E' or longitude_direction == 'e':

        longitude = longitude_degrees+longitude_minutes/60 +longitude_seconds/3600

    elif longitude_direction == 'W' or longitude_direction == 'w':

        longitude = -(longitude_degrees+longitude_minutes/60 +longitude_seconds/3600)

   

    latitude_longitude = [latitude, longitude]

   

    # Returns a list with both latitude and longiude in decimal format.

    return latitude_longitude

   

# Pulls the date information from

def get_date(image):

    # Gets all the exif data from the photo

    exif = {

        PIL.ExifTags.TAGS[k]: v

        for k, v in img._getexif().items()

        if k in PIL.ExifTags.TAGS

    }

    # Pulls the date and time from the exif format

    date = exif.get('DateTime').split()[0]

    time = exif.get('DateTime').split()[1]

    # Reformats the date to use - instead of :

    for character in date:

        if character == ':':

            date = date.replace(character, '-')

    # Combines the date and time to match the format pyinaturalist wants,

    date_time = str(date) + 'T' + str(time)

    # returns a date and time formatted to submit to iNaturalist with

    # pyinaturalist

    return date_time

# This presumes the name of the folder starts with the taxon number.It finds

# the taxon number by looking at the folder name and taking all the digits it

# sees. This allows you to name the folder "##### species name" to quickly

# tell where photos go. For example anything in '52381-Aphididae' is uploaded

# as an aphid.

def get_taxon(folder):

    taxon = ''

    folder =os.path.split(os.path.dirname(folder_name))[-1]

    for character in folder:

        if character.isdigit():

            taxon = taxon + character

    return taxon

# This is getting a token to allow photos to be uploaded.

token = get_access_token(username=user, password=passw,

                         app_id=app,

                         app_secret=secret)

# This goes to every file, checks if it is a jpg, gets the gps coordinates,

# get the time, and uploads it to iNaturalist.

for file in file_paths:

   if file[-3:] == 'jpg' or file[-3:] == 'JPG' or file[-3:] == 'Jpg':

       print('Uploading ' + file)

       try:

           img = PIL.Image.open(file)

           coordinates = get_lat_long(img)

       except:

           coordinates = 'No Coordinates'

       try:

           img = PIL.Image.open(file)

           date_time = get_date(img)

       except:

           date_time = 'No Date or Time' 

          

       # This requires the folder name to start with the taxon number.

       taxon = get_taxon(folder_name)   



       params = {'observation':

                    {'taxon_id': taxon,  # Vespa Crabro

                     'observed_on_string': date_time,

                     'time_zone': time_zone,

                     'description': '',

                     'tag_list': '',

                     'latitude': coordinates[0],

                     'longitude': coordinates[1],

                     'positional_accuracy': 50, # meters,

       

                     'observation_field_values_attributes':

                        [{'observation_field_id': '','value': ''}],

                     },}

       r = create_observations(params=params, access_token=token)

       

       new_observation_id = r[0]['id']

      

       from pyinaturalist.rest_api import add_photo_to_observation

       r = add_photo_to_observation(observation_id=new_observation_id,

                        file_object=open(file, 'rb'),

                        access_token=token)

print("Program complete")

Posted on 04 de fevereiro de 2019, 02:14 AM by glmory glmory | 4 comentários | Deixar um comentário

11 de fevereiro de 2019

Updated Python Upload Script

After a few rainy days I managed to come up with a python script using pyinaturalist which is at least as efficient as the existing upload method. For people uploading a large number of photos of the same species it is much more efficient.

The basic workflow is to put all the photos in a folder with the common name or scientific name and/or taxon number as the name of the folder.



You can put dozens of photos in any of these folders, so uploading fifty observations of the same species only requires running the script once. All the photos in every folder in the master folder will be uploaded as an individual observation.

What about uploading multiple photos to the same observation? This isn't much harder, add the photos to a sub-folder in the species folder. The script doesn't care what name, so I usually just leave these as "new folder." All the photos in this subfolder will be uploaded to an individual observation.

When you run the script, it gives you a few inputs to fill out:



The more annoying of these are "APP ID" and "APP Secret." You have to create an app for iNaturalist to upload through scripts. Fortunately this just takes a few seconds. The folder it wants is the main folder which contains all the species folders to be uploaded. Even if you are uploading just one species the folder with the photos will need to be in a master folder containing nothing but species folders to be uploaded.

Chances are you will want to go in the code and fill out default values for most of the entries so you will not have to fill them out every time you run the script. It is commented where to add them to the import_gui.py file.

Once the script is done, it moves all the files out of this folder, and puts them next to the main folder in a folder called "Uploaded." This should keep you from re-uploading everything if you lose connection mid-upload. Just run the script again, all the uploaded photos will now be gone. 

Since it got rather long, I uploaded it to Github as iNaturalist-Uploads. There are three files which all must be in the same folder. upload_folders.py is the file which is run as a python script. The other two (import_gui.py and import_functions.py) have functions which I preferred to put in a different folder to keep it less messy. All three files need to be saved to the same folder to run.

This probably doesn't make sense for most users as it is way less intuitive than the site submission tool, but if you are experienced with python or have a ton of photos of a limited number of species to upload this starts to make sense.

Posted on 11 de fevereiro de 2019, 05:19 AM by glmory glmory | 6 comentários | Deixar um comentário