Firefox History stats with Bash

25-09-2016 | Remy van Elst


Table of Contents


This is a small script to gather some statistics from your Firefox history. First we use sqlite3 to parse the Firefox history database and get the last three months, then we remove all the IP addresses and port numbers and finally we sort and count it.

#!/bin/bash
if [[ ! -d ${HOME}/.www ]]; then
  mkdir ${HOME}/.www/
fi

cp "$(find "${HOME}/.mozilla/firefox/" -name "places.sqlite" | head -n 1)" "${HOME}/.www/places.sqlite"
sqlite3 "${HOME}/.www/places.sqlite" "SELECT url FROM moz_places, moz_historyvisits \
                       WHERE moz_places.id = moz_historyvisits.place_id \
                             and visit_date > strftime('%s','now','-3 month')*1000000 ORDER by \
                       visit_date;"  > "${HOME}/.www/urls-unsorted"
sort -u "${HOME}/.www/urls-unsorted" > "${HOME}/.www/urls"

awk -F/ '{print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' -e ':.*' -e '^$' | sed -e 's/www\.//g' |sort | uniq -c | sort -n 

Example output:

    383 lowendtalk.com
    534 reddit.com
    569 google.com
    574 github.com
    792 encrypted.google.com
    973 i.imgur.com
   1458 next.duckduckgo.com
   3009 duckduckgo.com
   6459 wiki.int
  12934 management.int

If you just want HTTPS domains, or just HTTP you can use awk to make that happen:

awk -F/ '$1 == "https:" {print $3}' "${HOME}/.www/urls" | grep -v -E -e '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | sort | uniq -c | sort

Tags: bash, firefox, history, sqlite, ubuntu,