Given a Windows Vista or newer IShellItem, how do i get the system image list icon index associated with that item?
For example (pseudo
Basically it doesn't seem that there's any easy method for this. It simply hasn't been provided in the API.
In your question you say "But the shell supports things besides files and folders in the filesystem.", which makes me think you have overlooked that  SHGetFileInfo does actually support using PIDLs directly (with the SHGFI_PIDL flag) - so it can be used on non-filesystem objects. If you still have the full PIDL this is the easiest way to get an icon index, otherwise something like this should hopefully work:
int GetIShellItemSysIconIndex(IShellItem* psi)
{
    PIDLIST_ABSOLUTE pidl;
    int iIndex = -1;
    if (SUCCEEDED(SHGetIDListFromObject(psi, &pidl)))
    {
        SHFILEINFO sfi{};
        if (SHGetFileInfo(reinterpret_cast<LPCTSTR>(pidl), 0, &sfi, sizeof(sfi), SHGFI_PIDL | SHGFI_SYSICONINDEX))
            iIndex = sfi.iIcon;
        CoTaskMemFree(pidl);
    }
    return iIndex;
}
 Or using Raymond Chen's suggestion:
int GetIconIndex(IShellItem item)
{
    Int32 imageIndex;
    PIDLIST_ABSOLUTE parent;
    IShellFolder folder;
    PITEMID_CHILD child;
    //Use IParentAndItem to have the ShellItem 
    //cough up the IShellObject and child pidl it is wrapping.
    (item as IParentAndItem).GetParentAndItem(out parent, out folder, out child);
    try
    {        
       //Now use IShellIcon to get the icon index from the folder and child
       (folder as IShellIcon).GetIconOf(child, GIL_FORSHELL, out imageIndex);
    }
    finally
    {
       CoTaskMemFree(parent);
       CoTaskMemFree(child);
    }
    return imageIndex;
}
Turns out that IShellFolder doesn't support the IShellIcon sometimes. For example attempting to browse inside a zip file. When that happens, the QueryInterface of IShellFolder for IShellIcon fails.
shellFolder.QueryInterface(IID_IShellIcon, out shellIcon); //<--fails with E_NOINTERFACE
Yet SHGetFileInfo knows how to handle it. 
So best to not try to get an IShellIcon interface yourself. Leave the heavy lifting to SHGetFileInfo (at least until someone from Microsoft documents how to use IShellIcon).
In your great investigation you forget about IShellIcon interface. It available even in Windows XP.
function GetIconIndex(AFolder: IShellFolder; AChild: PItemIDList): Integer; overload;
var
  ShellIcon: IShellIcon;
  R: HRESULT;
begin
  OleCheck(AFolder.QueryInterface(IShellIcon, ShellIcon));
  try
    R := ShellIcon.GetIconOf(AChild, 0, Result);
    case R of
      S_OK:;
      S_FALSE:
        Result := -1; // icon can not be obtained for this object
    else
      OleCheck(R);
    end;
  finally
    ShellIcon := nil;
  end;
end;