I had a closer look at the dynamic dma interface
introduced in 2.3.41 and, although a step in the
right direction, it's very much tailored to just
sparc64 and solving only the problem of 32 bit
dma's in a .gt. 32 bit paddr space. I think
without much impact on their existing sparc64
code, this could be made a lot more general.
Description
-----------
A brief summary of the interfaces as I understand
them is:
Following two routines are used for what they call
"static" dma mappings. These are roughly equivalent
to PCIIO_DMA_CMD type mappings in Irix. The 'alloc'
routine will allocate the memory, acquire necessary hw
resources and map the vaddr to a paddr and return this
value. The 'free' routine free's hw resources and
the allocated memory.
void *pci_alloc_consistent(struct pci_dev *pdev,
size_t size,
dma_addr_t *dma_addrp)
void pci_free_consistent(struct pci_dev *pdev,
size_t size, void *cpu,
dma_addr_t dvma)
These four routines are used for what they call
"streaming" dma mappings. These are roughly equivalent
to PCIIO_DMA_DATA type mappings in Irix. The routines
assume the device driver has allocated the required
memory and they simply do the mapping. The 'sg'
routines are used for scatter-gather dma's.
dma_addr_t pci_map_single(struct pci_dev *pdev,
void *addr,
size_t sz)
void pci_unmap_single(struct pci_dev *pdev,
dma_addr_t bus_addr,
size_t sz)
int pci_map_sg(struct pci_dev *pdev,
struct scatterlist *sglist,
int nelems)
void pci_unmap_sg(struct pci_dev *pdev,
struct scatterlist *sglist,
int nelems)
Finally, these two routines are used in conjunction with
the 'pci_map_*' routines to force memory consistency
if you use the same streaming dma region multiple times
and touch the data between dma transfers.
void pci_dma_sync_single(struct pci_dev *pdev,
dma_addr_t bus_addr,
size_t sz)
void pci_dma_sync_sg(struct pci_dev *pdev,
struct scatterlist *sglist,
int nelems)
Issues
------
1. Assumes that resources are only required for 32-bit devices doing
dma's into a .gt. 32-bit paddr space. On mips64 we need to allocate
resources for all dma's (i.e. RRB's on the bridge are required for
any dma, and in addition ATE's are required for 32-bit dma's).
How do we distinguish between 32bit dma's and 64bit dma's with this
interface? Jakub states outright in one of his postings that
the "whole interface is currently for SAC".
2. Not what I consider an orthogonal interface. E.g. pci_alloc_consistent()
will actually call page_alloc() for you whereas pci_map_single() expects
you to already have allocated memory. I makes more sense to let the
driver explicitly do the memory allocation. Then this interface needs
only allocate any hw resources for the dma and then do the mapping.
This also allows you to reserve a much larger mapping size than
what you may want to actually allocate, which would be useful
if the reservation of hw resources were decoupled from the actual
mapping of addresses.
3. The interface is not very extendible. Different kinds of mappings
are provided through api, so extensions are only possible through
new api.
Proposal
--------
The issues above could be addressed fairly easily. First of all,
there's no need for pci_alloc_consistent() to allocate the memory.
Instead let the driver allocate the memory. Then rather than
have a pci_alloc_consistent() call, just have a pci_map_single()
call which takes a flag argument. The flag could specify
CONSISTENT vs. STREAMING. This would make it easier for mips64
to add additional performance-oriented flags that make sense for
different platforms (and would be no-ops on unsupported hardware).
Most importantly it allows you to specify (through a flag) when your
card is 64 bit capable, so the same interface can work for either 32 bit
or 64 bit pci cards.
Ideally we would want to distinguish between allocating hw resources
and actually translating (i.e. mapping) addresses. The
pci_map_single() interface ties the two together. There's
some performance to be gained by separating the two because
that allows you to grab the hw resources just once and
reuse them for different dma's. Also it means that you
don't need a flag argument for the 'sg' interfaces (assuming
you were going to use a flag to distinguish between 32-bit
and 64-bit pci cards). You simply have one
pci_alloc_resources() call which takes the flag argument,
allocates the appropriate resources, and then
pci_map_single/pci_map_sg do the appropriate address
mapping.
Comments?
Btw, if you want to see the code, I downloaded 2.3.41 to
annie.engr:/usr/src/linux-2.3.41. The interfaces are coded
in arch/sparc64/kernel/pci_iommu.c
- leo
--
Leo Dagum SGI Mountain View, CA 94043 (650-933-2179)
|