Previously, in my Self-hosted finances series, I cleaned and identified transfers in my Mint transactions for the purposes of of importing into Firefly-iii. In this post, I’m going to import the transactions into Firefly-iii.
This part is comparatively easy vs the previous steps, however it’s only a one time import. A continue updating workflow is tricky and I’m working on some logic to do that.
Authenticating
First, we need a token that allows us to call the API. To get this, login to Firefly, and open Options, then Profile.
Then go to OAuth and click Create a new token
Firefly will then show you a token. Copy and store this in the script as a variable and also specify the endpoint where Firefly is being served:
1
2
| fireflyToken = 'eYJ...'
host = 'https://firefly.example.com'
|
Constructing the request
This next method will translate the DataFrame created in the previous post request into the format that the Firefly API expects and creates the transaction.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
| import json
import requests
session = requests.Session()
def send_to_firefly(record):
item = {
'type': record['type'],
'amount': record['amount']
}
for key in ['description', 'destination_name', 'source_name', 'category_name', 'notes']:
if key in record and not pd.isnull(record[key]):
item[key] = record[key]
for key in ['source_id', 'destination_id']:
if key in record and not pd.isnull(record[key]) and record[key] != '':
item[key] = str(int(record[key]))
item['foreign_amount'] = '0'
item['reconciled'] = False
item['order'] = '0'
# If you use something other than USD, change this value
item['currency_id'] = '8'
item['currency_code'] = 'USD'
for key in ['date', 'process_date']:
if key in record and not pd.isnull(record[key]) and record[key] != '':
item[key] = record[key].isoformat()
if isinstance(record['tags'], str):
item['tags'] = [record['tags']]
else:
item['tags'] = []
payload = {
"transactions": [
item
],
"apply_rules": True,
"fire_webhooks": False,
"error_if_duplicate_hash": False
}
try:
json.dumps(payload)
except Exception as e:
raise Exception(f"Can't serialize '{item}: {e}")
resp = session.post(f"{host}/api/v1/transactions", headers={"Authorization": f"Bearer {fireflyToken}", "Content-Type": "application/json", 'Accept': 'application/json'}, json=payload, allow_redirects=False)
if resp.status_code == 200:
return pd.Series([resp.status_code, resp.json(), resp.json()['data']['id']], index=['status', 'message', 'firefly_id'])
else:
return pd.Series([resp.status_code, resp.json()], index=['status', 'message'])
|
Sending the request
Next, it’s easy to send the requests to Firefly:
1
2
3
4
5
| # First find the transfers and translate into common format
output = transactions.apply(func=process_record, axis=1, result_type='expand')
# Then send it to Firefly
fireflyResult = output[output['amount'] != 0].apply(func=send_to_firefly, axis=1, result_type='expand')
|
Reviewing Results
Firefly can return non-200 status code responses. Running this will identify all the non-successful writes. In theory, this should empty. It’s a good idea to check the error messages and see what’s wrong.
1
| print(fireflyResult[(fireflyResult['status'] != 200)])
|
Conclusion
Once we’ve identified the transfers, it’s comparatively easy to write the one-time extract to Firefly. The code can be found in my GitHub repository. I’ve got a few projects in progress right now, including tools to automatically scrape data from accounts and how to merge new transactions into Firefly.
Comments
To give feedback, send an email to adam [at] this website url.
Other Posts in Series
This post is part of the Self-hosted Finances series. You can check out the other posts for more information:
Donate
If you've found these posts helpful and would like to support this work directly, your contribution would be appreciated and enable me to dedicate more time to creating future posts. Thank you for joining me!
