Testing Firefox Update

* There are two kinds of update check:
    1) Updating to a newer version of an Extension or Theme
    2) Updating an existing Extension or Theme with include newer 
       application compatibility information from a remote service
       (either the update.mozilla.org service, a custom VersionCheck service
        installation or a custom update RDF file).
       This is the case when an Extension author decides that his Extension
       works with a newer version of Firefox after shipping it and increases
       maxVersion on his website or in the umo database so that existing 
       installs can work when users upgrade to the newer version of Firefox.

* Compatibility Checking
  When a user upgrades to a newer version of Firefox, all of their extensions
  that do not advertise a compatibility range that includes the newer version
  number are disabled and on first launch the Mismatch UI is presented. 
  
  The goal of the Mismatch UI is to discover and apply updates from the web
  to incompatible installed Extensions and Themes, and minimize UI as much
  as possible since software installation is already a traumatic experience.
  For this reason, two kinds of check are performed. First a check of type
  (2) is performed to see if the incompatible Extensions and Themes are 
  actually compatible. If all the items are compatible and only version 
  patching is necessary, the items' version info is patched in the EM 
  datasource and the wizard closes and the app continues starting.
  
  If there are items that version patching is not available for, Firefox
  must check for newer XPIs to download and install that are compatible.
  Firefox performs this check after the Version patching check, and displays
  available XPIs to download. The user can then get what ones they want. 
  
* Things You Should Know

  This document makes reference to several files:
  
  "compatibility.ini" - in your profile, I'll call it "FCOMPAT" for short.
  "Extensions.rdf" - in profile/extensions/, I'll call it "EXTDB"
  "prefs.js" - in your profile dir, I'll call it "PREF" for short.

Set Up VersionCheck

- TODO: instructions for setting up VersionCheck. An instance of VersionCheck
        is required for many of these tests.

Mismatch Checking: Version Patches

1) Create a New Profile
2) Run Firefox
3) Install newext5-31.xpi and newext6-90.xpi
4) Close Firefox and restart
6) Go to about:config and set app.extensions.version to "0.12"
7) Clear extensions.lastAppVersion
8) Close Firefox
9) Open EXTDB and set maxVersion for both to "0.9"
10) Open FCOMPAT and increment the Build ID field
11) Start Firefox
12) The Mismatch Update Wizard should appear, first applying
    version patches. If the services are configured properly it
    should vanish after a few seconds and Firefox should start.
    Neither NewExt5 or NewExt6 should be disabled, both should
    be fully functional after retrieving version updates from
    the VersionCheck service. 
13) To run this test again without starting from the top:
    1) open EXTDB and reset maxVersion for both to "0.9"
    2) open FCOMPAT and increment the Build ID field again
    3) open PREF and remove the line for extensions.lastAppVersion
    4) start Firefox, Mismatch Checking will start again

Mismatch Checking: Version Updates

1) Create a New Profile
2) Run Firefox
3) Install newext5-30.xpi
4) Close Firefox and restart - verify 3.0 is installed.
6) Go to about:config and set app.extensions.version to "0.12"
7) Clear extensions.lastAppVersion
8) Close Firefox
9) Open EXTDB and set maxVersion to "0.9"
10) Open FCOMPAT and increment the Build ID field
11) Start Firefox
12) The Mismatch Update Wizard should appear, first trying to 
    locate version patches (there are none - the only remote
    version is newext5-31, not newext5-30). Once it fails to
    find any it'll try and find newer versions. In this case
    it will find the 31 upgrade and show newext5-3.1. 
13) Select newext5-3.1 and click Install
14) Firefox should download and install the update. Once
    Firefox is fully started verify 3.1 is installed and fully
    functional. 
15) To run this test again without starting from the top:
    1) open EXTDB and reset maxVersion for the extension to "0.9"
       and the version field to "3.0"
    2) open FCOMPAT and increment the Build ID field again
    3) open PREF and remove the line for extensions.lastAppVersion
    4) start Firefox, Mismatch Checking will start again

Mismatch Checking: Combination

1) Create a New Profile
2) Run Firefox
3) Install newext5-30.xpi and newext6-90.xpi
4) Close Firefox and restart
6) Go to about:config and set app.extensions.version to "0.12"
7) Clear extensions.lastAppVersion
8) Close Firefox
9) Open EXTDB and set maxVersion for both to "0.9"
10) Open FCOMPAT and increment the Build ID field
11) Start Firefox
12) The Mismatch Update Wizard should appear, first applying
    version patches. It will find a version patch for 6-90
    and should apply that and then show the incompatibility
    details page for 5-30 since no version patch was available
    for that one. 6-90 should not show up in that list.
    Click Next to check for updates and the update to 5-31 
    should appear. 
13) To run this test again without starting from the top:
    1) open EXTDB and reset maxVersion for both to "0.9"
    2) open FCOMPAT and increment the Build ID field again
    3) open PREF and remove the line for extensions.lastAppVersion
    4) start Firefox, Mismatch Checking will start again

NEEDED TESTS:
- Extension XPIs with no available update (one that uses VersionCheck,
  one that uses custom RDF)
- Extension XPIs with only one update whose minVersion exceeds the new
  version installed (one using VersionCheck, one using Custom RDF)
- Extension XPIs with several available updates, some of which are
  compatible with the newly installed version, some which exceed the
  new version, some which are older. (XPIs using Versioncheck, XPIs
  using Custom RDF)

Installation: Incompatible With Compatible Phone Home Update

- Install newext8-50.xpi
- newext8-50.xpi is incompatible with 0.9 but should install anyway
  as there is version update info on the VersionCheck service
  
Installation: Incompatible

- Install newext9-50.xpi
- newext9-50.xpi is incompatible and there is no update available. 
  It should display an error informing you of this.
  
Installation: Incompatible with Incompatible Phone Home Update

- Install newext10-50.xpi
- newext10-50.xpi is incompatible and has an incompatible update
  It should display an error informing you that it is incompatible.
  
NEEDED TESTS:
- variations on these themes, custom RDF vs VersionCheck, etc.

Background Update:

- Every so often, on an interval defined in the pref update.interval, 
  Firefox checks to see if the length of time since its last update
  exceeds the interval for that update type. More simply, by default:
  
  Every hour Firefox checks to see if it has been one day since
  it last checked for a new version of itself, and
  
  Every hour Firefox checks to see if it has been one week since it
  checked to see if there are any new versions of installed Extensions/
  Themes. 
  
  You can control the timer (the hour value) by setting update.interval. 
  You can control the application update frequency by setting 
  update.app.interval.
  You can control the extension update frequency by setting 
  update.extensions.interval.
  
  Every time an update check is performed, Firefox writes the current
  time in UTC seconds to update.extensions.lastUpdateDate/
  update.app.lastUpdateDate. 
  
  When the hourly timer fires, Firefox makes this decision:
  
  Do an update check if:
  
  now (in seconds) - last update time (in seconds) > update interval
  
  Knowing these prefs and their meanings you can test automated 
  update. Install some of the older versioned Extensions supplied in 
  the test suite with the prefs configured so that you don't need to
  wait hours or days, then note the notification UI. 
  
  The notification UI should show an icon representing the severity
  of the available updates, and a count of available updates.
  
  The icon is:
  
  - A red (flashing) "Important" icon if there is a newer version of 
    Firefox available, regardless of how many Extension or Theme
    updates are available...
    
    If there is no new version of Firefox available, then:
    
  - A yellow triangle icon if there are more than 
    update.extensions.severity.threshold updates to Extensions and
    Themes available (5 by default, set this to a lower value like 2
    to test with this test suite)
  - A subtle info icon if there are less than the threshold updates
    available.
    
  Preferences:
  
  - update.app.enabled and update.extensions.enabled are boolean
    prefs set in the Advanced panel of Options that determine whether
    or not Firefox should background-check for these things 
    periodically.
  
  - regardless of how these prefs are set, users should be able to manually
    check for updates. 

Manual Update:

- Install lower versions of any of the test suite and then either:
  1) open the Extension manager and click the item then click 
     Update to search for updates for that item.
  2) open the Extension manager and clear the selection by clicking
     in a blank area and then click Update to search for updates to
     all items. 
  3) double click the notification area in a browser window to
     search for updates to the application and extensions/themes.

NEEDED TESTS:
- lots of simple extensions that test the various kinds of updates 
  possible.
- tests of extensions that make mistakes or abuse the API, with 
  suggestions on how to handle, recover