xenocara/xserver/hw/xfree86/doc/devel/Domain.note

The purpose of the changes described here is to implement a more general
framework for multi-head on systems with more than one host-to-PCI bridge.
The changes also implement a basic port of XFree86 to SPARC Solaris.

These changes are derived from David S. Miller's submission #4653 to the
patch list.  David Andrew of Sun Microsystems was also kind enough to
arrange for a hardware loan for development of these changes.

These changes are known to work on several SPARC SunOS and UltraSPARC
Linux configurations.  Linux kernel work is in progress to port these
changes to Linux/PowerPC.

Several loose ends still need to be addressed before these changes can be
considered stable.  The bulk of this note is devoted to enumerating what
remains to be done, along with other notes, broken down into various broad
categories.

SPARC SunOS (aka Solaris)
-------------------------
- An overview of this XFree86 port is available in README.Solaris.
- The keyboard map code in hw/xfree86/os-support/sunos/sun_kbdEv.c needs
  to be extended to handle more than only the sun5 keyboard I targeted it
  for.  Even for the sun5, the map is incomplete as several keys are not
  mapped.  What is there is just barely usable.
- On exit, the server will zero out /dev/fb, but that might not be the
  right thing to do for all primary adapters.  This does however
  appear to emulate the behaviour of Sun's commercial servers.  It also
  eliminates the need for output drivers to save and restore video memory
  contents.  (They still need to save/restore the mode timing however.)
  This also chimes into a long-standing XFree86 policy to not save/restore
  video memory contents if the mode on entry is found to be non-VGA, a
  policy several existing drivers comply with.
- The SBUS drivers (sunbw2, suncg14, suncg3, suncg6, sunffb, sunleo and
  suntcx), the common layer's SBUS code and the fbdev driver have all
  only been compile tested.  There are likely to be Linux'isms within
  them that remain to be dealt with.
- It still needs to be verified whether or not this work adversely
  affected support for ix86 Solaris.

UltraSPARC Linux
----------------
- Although this code can be compiled using any Linux/SPARC64 kernel, it
  can only run successfully using 2.4.12 or later.
- I haven't had time to sufficiently dig into XKB to properly configure it
  for sun5 keyboards.  Given XFree86 on Linux/SPARC has been around for a
  while, it's likely someone has already done this, and I'd appreciate
  receiving a copy of a working XF86Config input section.

PowerPC Linux
-------------
- As mentioned above, kernel work is in progress to port this PCI scheme
  to Linux/PowerPC.
- Aside from kernel work, the inX() and outX() definitions in compiler.h
  will need to be changed to do something akin to their SPARC definitions,
  i.e. consider their port argument to be a virtual address.

Other Linux ports to multi-domain architectures
-----------------------------------------------
- Comments in os-support/bus/linuxPci.c document the kernel interface
  required to port these changes.  In short, Linux ports, such as Alpha
  and mips, should follow SPARC and PowerPC's lead in providing support to
  mmap() PCI devices through their /proc/bus/pci pseudo-files and to treat
  such requests for host bridges as requests to mmap() space provided by
  these bridges.

Other OS's
----------
- In the right hands, either linuxPci.c or sparcPci.c can be used as a
  guide for what would need to be done to port this scheme to other OS's.
  Perhaps the largest difference between the two (in terms of interface to
  the common layer) is that the SunOS port includes internally generated
  domain numbers in PCITAG's, whereas the Linux port doesn't need to.  The
  remainder of the PCI code (which is OS-independent) can handle either
  scheme.
- Required entry points are xf86GetPciDomain(), xf86MapDomainMemory(),
  xf86MapDomainIO() and xf86ReadDomainMemory().  Replacements for
  xf86BusAccWindowsFromOS(), xf86PciBusAccWindowsFromOS() and
  xf86AccResFromOS() might also be required.
- Development of these changes has detected the fact that the XFree86 port
  to the PowerMax OS is broken, and has been for some time, i.e. since
  shortly after its introduction, back in the 3.9* days.

SPARC PCI (OS-independent)
--------------------------
- The "Simba" PCI-to-PCI bridge used in SPARC's does not implement VGA
  routing, as defined in the PCI specs.  Fortunately, OpenPROM seems to
  always route VGA resources to the bus with PCI connectors, but this also
  causes the common layer to not mark any PCI adapter as primary.

Multiple PCI domains (architecture- and OS-independent)
-------------------------------------------------------
- This implementation assumes every host-to-PCI bridge provides access to
  a separate PCI domain.  Each such domain provides three different
  "address" spaces:  PCI configuration, I/O and memory.  The
  implementation can also deal with situations where more than one PCI
  domain share (different subsets of) the same PCI configuration space.  I
  have unconfirmed information that suggests it might be necessary to also
  allow the sharing of PCI memory spaces.
- This implementation also assumes the CPU's physical address space
  includes the entirety of each domain's I/O and memory spaces.  I know
  this'll need to be changed to deal with the so-called UniNorth bridge,
  found on PowerPC's, which allows access to only a subset of the memory
  space behind it.
- Ideally, the common layer should mark as primary up to one PCI adapter
  per domain.  This has yet to be done.
- Something needs to be done about PCI master aborts on primary buses.
  For details on this, see my long-winded diatribe in sparcPci.c, and
  related comments in linuxPci.c.  Suffice it to say here that I see the
  eventual implementation of host bridge drivers within XFree86 as
  unavoidable at this point.
- DGA is broken on multi-domain platforms.  The information passed to the
  client to locate the framebuffer still needs to be revised.  The best way
  to deal with this is to change all drivers' OpenFramebuffer() function to
  call a common layer routine to set the device name and displacements to be
  returned to the DGA client.

Output drivers
--------------
Most drivers currently used on ix86 need(ed) source code changes.
- Calls to xf86ReadBIOS() and xf86MapVidMem() were replaced with calls to
  xf86ReadDomainMemory() and xf86MapDomainMemory() respectively.  Except
  for the "ati" and "atimisc" modules, this has already been done.
- All ix86-style I/O port numbers need to be declared as an IOADDRESS, a
  type defined in xf86Pci.h as "unsigned long".  Such port numbers also
  need to be offset by a displacement which is also defined as an
  IOADDRESS.  Before a driver's PreInit() is called, the common layer
  makes this displacement available in ScrnInfoRec.domainIOBase.  For
  single-domain architectures, such as ix86, domainIOBase will always be
  zero.  Current use of vgaHWRec.PIOOffset has also been adjusted
  accordingly.  Some drivers have been changed to keep a copy of this
  displacement in their private structure.  Internally, an IOADDRESS is
  actually a pointer that has been recasted to an unsigned long, but the
  common layer "hides" this fact from the driver ABI, which means that I/O
  port numbers, as seen by drivers, remain as integers rather than
  addresses.  Aside from the ati and atimisc modules, s3, sis and tseng
  are the only modules left whose I/O still needs to be converted (I've
  temporarily run out of steam).
- Note that these conversions are not necessarily sufficient to produce
  drivers that will work on any given multi-domain architecture.  A driver
  that, for example, had endianness problems, still does.  But, at least,
  these conversions, along with the supporting common layer changes, make
  PCI drivers more widely amenable to porting.
- rdinx(), wrinx(), modinx(), testrg(), testinx() and testinx2() are not
  given enough information to allow for the relocation of their I/O.  They
  are consequently being deleted.  The apm and ark drivers, the only
  remaining callers of the first three, have been changed to use local
  definitions instead.  The last three (test*()) were already unused.
- As a temporary measure, these changes completely disable ISA-style
  probing on SPARC's and PowerPC's.  This means that driver calls to
  xf86MatchIsaInstances(), while still valid, will always return detection
  failure on SPARC's and PowerPC's.  This will be dealt with when a more
  general master abort handling scheme is implemented.
- I need to make a decision about the master abort issues mentionned above
  before I can convert the "ati" and "atimisc" modules.  Consequently,
  these modules still need to be compiled with -DAVOID_CPIO on
  multi-domain architectures, and support for Mach64 variants as
  non-primary heads is not yet available.

$XFree86$