Altium Unified Components - Catch 'Em All!

Until recently, the Altium Designer "official" component libraries (known as "Unified Components") were only available to Altium subscribers - they are now however available to anyone by filling out a short survey (no registration or subscription required). After doing this, you receive an email with a download link. This process gets old and tedious very quickly every time you need a new component library... Yes, that's right: Altium does not offer a single download for all the libraries! So I decided to take matters into my own hands and wrote a Python script that uses some XPath and Selenium magic to automatically download all 578 libraries in less than 5 minutes :) This script actually relies on the fact that the library download links are contained in the page source! Thanks Altium!

Requirements:

• Python 3.4.4
• PhantomJS 2.1.1
• Selenium 3.0.2

Usage (Windows):
1. Download and install Python.
2. Install Selenium:

pip install selenium

3. Download the PhantomJS pre-built executable and place in the same directory as script.
4. Run Python script:

python download_all_altium_unified_components.py

I am of course providing the script source for educational purposes only... :)
Any suggestions or feedback on this script are welcome too.

download_all_altium_unified_components.py

from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import re
import concurrent.futures
import urllib.request
import urllib.parse
import posixpath
import os

BASE_URL                = "https://designcontent.live.altium.com/"
START_URL               = BASE_URL + "UnifiedComponents/"
PAGE_LOAD_WAIT_S        = 20
DOWNLOAD_TIMEOUT_S      = 60
MAX_DOWNLOAD_WORKERS    = 8

def wait_for_page_to_load(driver, timeout_s):
    WebDriverWait(driver, timeout_s).until(EC.presence_of_element_located((By.ID, "xUCS_frmItemsView")))

def get_next_page_url(driver):
    try:
        link = driver.find_element_by_xpath("//div[@id='xUCS_frmItemsView']//div[@id='UCS_frmItemsView_TextLabel2']/div/div[1]/following-sibling::a[1][@class='HijaxLink']")
        return BASE_URL + link.get_attribute("href").replace(START_URL, "")
    except:
        return ""

def get_content_urls(driver):
    divs = driver.find_elements_by_xpath("//div[contains(@id, 'UCS_frmItemsView_Fields') and @id!='UCS_frmItemsView_FieldsPopup']")
    return [re.search('ZipURL`(.*\.zip)`[\S+\n\r\s]+', div.get_attribute("title")).group(1) for div in divs]

def download(url, timeout):
    with urllib.request.urlopen(url, timeout=timeout) as conn:
        return conn.read()

# Start WebDriver (choose one!)
print("Starting WebDriver...")
#driver = webdriver.Firefox()
driver = webdriver.PhantomJS(service_args=['--load-images=no'])
print("WebDriver started.")

# Open page
driver.get(START_URL)
# Wait for page to load
wait_for_page_to_load(driver, PAGE_LOAD_WAIT_S)
# Get URLs
urls_all = list()
urls = get_content_urls(driver)
urls_all += urls
print("%r: ZIP download links retrieved from page: %d (Total: %d)" % (driver.current_url, len(urls), len(urls_all)))

# Do remaining pages
next_page_url = get_next_page_url(driver)
while next_page_url != "":
    # Open page
    driver.get(next_page_url)
    # Wait for page to load
    wait_for_page_to_load(driver, PAGE_LOAD_WAIT_S)
    # Get URLs
    urls = get_content_urls(driver);
    urls_all += urls
    print("%r: ZIP download links retrieved from page: %d (Total: %d)" % (driver.current_url, len(urls), len(urls_all)))
    # Get next page URL (if any)
    next_page_url = get_next_page_url(driver)

# Quit WebDriver
driver.quit()

# Write URLs to file
urls_file = open('altium_unified_components_urls.txt', 'w')
for url in urls_all:
    urls_file.write("%s\n" % url)
urls_file.close()

# Download all URLs
error_count = 0
print("%d files will be downloaded..." % len(urls_all))
with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_DOWNLOAD_WORKERS) as executor:
    # Start the download operations and mark each future with its URL
    future_to_url = {executor.submit(download, url, DOWNLOAD_TIMEOUT_S): url for url in urls_all}
    for future in concurrent.futures.as_completed(future_to_url):
        url = future_to_url[future]
        try:
            data = future.result()
            print('%r file is %d bytes' % (url, len(data)))
            # Get filename from URL
            path = urllib.parse.urlsplit(url).path
            filename = urllib.parse.unquote(posixpath.basename(path))
            # Save file
            savepath = "download/" + filename
            os.makedirs(os.path.dirname(savepath), exist_ok=True)
            with open(savepath, "wb") as file:
                file.write(data)
        except Exception as e:
            print('%r generated an exception: %s' % (url, e))
            error_count += 1

# Done!
if error_count == 0:
    print("Finished successfully.")
else:
    print("Finished with %d error(s)." % error_count)

Update 1 (June 25 2017):
Download link for Unified Components archive as obtained from the above script. Enjoy!

Update 2 (December 9 2017):
14 new libraries have since been added by Altium bringing the total to 592 (updated library archive download link). New libraries:
TE_Connectivity_ D-Shaped_Connectors.zip
TE_Connectivity_Audio,_Video_and_High_Speed_Serial_Connectors.zip
TE_Connectivity_Automotive_Connectors_(Car,_Truck,_Bus,_and_Off-Road).zip
TE_Connectivity_Card_Edge_Connectors_and_Sockets.zip
TE_Connectivity_Humidity_Sensors.zip
TE_Connectivity_Modular_Jacks_and_Plugs_(RJ45,_RJ11,_RJ25).zip
TE_Connectivity_Passive_Components.zip
TE_Connectivity_PCB_Connectors.zip
TE_Connectivity_PCB_Terminals.zip
TE_Connectivity_Power_Connectors,_Contacts,_and_Terminals.zip
TE_Connectivity_Pressure_Sensors.zip
TE_Connectivity_Relays,_Contactors_and_Switches.zip
TE_Connectivity_RF_and_Coax_Connectors.zip
TE_Connectivity_Terminal_Blocks_and_Strips.zip  

Update 3 (January 13 2018):
I have slightly modified the script to download all available online content: Unified Components, NanoBoard Examples and Reference Designs (Note: Template Designs seem to only be available via Altium Vault).
download_altium_design_content.py
altium_design_content_20180113.zip [888 libraries, 1.34 GB]