BioMart - biological queries made easy
© Smedley et al. 2009
Received: 22 September 2008
Accepted: 14 January 2009
Published: 14 January 2009
Skip to main content
© Smedley et al. 2009
Received: 22 September 2008
Accepted: 14 January 2009
Published: 14 January 2009
Biologists need to perform complex queries, often across a variety of databases. Typically, each data resource provides an advanced query interface, each of which must be learnt by the biologist before they can begin to query them. Frequently, more than one data source is required and for high-throughput analysis, cutting and pasting results between websites is certainly very time consuming. Therefore, many groups rely on local bioinformatics support to process queries by accessing the resource's programmatic interfaces if they exist. This is not an efficient solution in terms of cost and time. Instead, it would be better if the biologist only had to learn one generic interface. BioMart provides such a solution.
BioMart enables scientists to perform advanced querying of biological data sources through a single web interface. The power of the system comes from integrated querying of data sources regardless of their geographical locations. Once these queries have been defined, they may be automated with its "scripting at the click of a button" functionality. BioMart's capabilities are extended by integration with several widely used software packages such as BioConductor, DAS, Galaxy, Cytoscape, Taverna. In this paper, we describe all aspects of BioMart from a user's perspective and demonstrate how it can be used to solve real biological use cases such as SNP selection for candidate gene screening or annotation of microarray results.
BioMart is an easy to use, generic and scalable system and therefore, has become an integral part of large data resources including Ensembl, UniProt, HapMap, Wormbase, Gramene, Dictybase, PRIDE, MSD and Reactome. BioMart is freely accessible to use at http://www.biomart.org.
In this post-genomics era, data of increasing volume and complexity is being deposited into databases around the world. Biologists need to ask complex queries of this data to test and drive their research hypotheses. Typically, each data source provides an advanced query interface on their website to satisfy this requirement. However, each site has its own solution and subsequently, the user has a learning curve before they can start interacting with the data. A further problem the researcher has is that they often need to query more than one data source, necessitating mastering more than one interface and having to cut and paste results between the sites. If the analysis involves high-throughput data, this approach is not usually scalable. To overcome this problem, many groups rely on bioinformaticians who can generate scripts to interact with the varying programmatic interfaces of the different data sources. They also often have to learn a number of different web services or application programmatic interfaces (APIs) for each resource. A preferable solution would be to have generic software that a biologist can use on top of any data source. BioMart is such a solution.
BioMart is an open source data management system that comes with a range of query interfaces that allow users to group and refine data based upon many different criteria. In addition, the software features a built-in query optimiser for fast data retrieval. A BioMart installation can provide domain-specific querying of a single data source or function as a one-stop shop (web portal) to a wide range of BioMarts as our central portal  does. All BioMart websites have the same look and feel (only varying in colour scheme and branding), which has obvious advantages to users moving between different resources. However, the power of the system comes from integrated querying of the different BioMarts. If any datasets share common identifiers (such as Ensembl gene IDs or Uniprot IDs) or even mappings to a common genome assembly, these can be used to link BioMarts together in integrated queries. Additionally, these datasets do not have to be located on the same server or even at the same geographical location. This distributed solution has many advantages; not least of which is the fact that each site can utilise their own domain expertise to deploy their BioMart.
BioMart also has the advantage of being integrated with external software packages such as BioConductor , the Distributed Annotation System (DAS) , Galaxy , Cytoscape , Taverna . This enables users to perform integrated queries with non-BioMart data sources as well as detailed analysis of the results. BioMart is also part of the GMOD (Generic Model Organism Database)  suite of tools for building a model organism site.
Description of all publicly accessible BioMarts to date
Name of BioMart
Description of contents
Location of BioMart
Automated annotation of over 40 eukaryotic genomes
Ensembl Compara orthologues and paralogues
Ensembl Variation data from dbSNP and other sources
Ensembl Genomic Features
Ensembl Markers, clones and contigs data
Manually curated human, mouse and zebrafish genes
High throughput gene targeting/trapping to produce mouse knock-outs
Comparative Grass Genomics
Curated database of biological pathways
C. elegans and C. briggsae genome database
Dictyostelium discoideum genome database
Northwestern University, USA
Rat model organism database
Medical College of Wisconsin, USA
Proteomic data repository
Rat tissue expression compendium
Protein sequence and function repository
Pancreatic Expression Database
Pancreatic cancer expression database
Barts & The London School of Medicine, UK
Peptide mass spectrometer data for proteomics
University of Manchester, UK
Microarray data repository
Cross species knowledgebase of genes relevant for sexual reproduction
Annotation of 12 Drosophila genomes
Indiana Univeristy, USA
Catalogue of common human variations in a range of populations
Invertebrate vectors of human pathogens
University of Notre Dame, USA
Paramecium tetraurelia model organism database
Mouse in situ expression data
MRC Edinburgh, UK
Mouse phenotype data from high throughput standardized screens
MRC Harwell, UK
The range of interfaces is designed with both biologists and bioinformaticians in mind. The simplest way of querying BioMart is via the web interface called MartView (either on our central portal  or follow the links on our main page  to the individual sites). Programmatic access is available via a Perl API or BioMart's web services (MartServices). An important and novel feature of BioMart is that it offers "scripting at the click of a button". A user can generate an API or MartServices script by building up a query on the MartView website followed by a simple click of a button. All the interfaces allow the user to build up biological queries by first selecting dataset(s) of interest, then the data to view and/or save (attributes), some optional restrictions (filters) on the query and finally the format for the data.
Here we will describe a top-level view of the BioMart system as the focus of this paper is on practical use of BioMart rather than implementation and deployment. Further documentation on these aspects can be found on the BioMart website http://www.biomart.org. BioMart is designed around a simple, three tier architecture:
(i) The first tier consists of one or more relational databases. Each database may consist of one or more "marts", which are schema compliant with BioMart definitions. Each "mart" may hold a number of different datasets. The BioMart data model is a denormalised, query optimised schema and can be deployed using Oracle, MySQL or Postgres relational database management systems. Each dataset uses a reversed star model  where data mapping 1:1 with the central object being modelled is stored in a main table. Data mapping 1:n is stored in one or more satellite, dimension tables. Two tools are provided to build and configure the mart databases in the first tier:
MartBuilder, to construct SQL statements that will transform your schema into a mart.
MartEditor, to configure the finished mart for use with the rest of the system. This produces a dataset configuration XML (Extensible markup language) that is stored in metadata tables within the actual mart database.
(ii) The second tier is the Perl API (distributed in the biomart-perl package) which interacts with both the dataset configuration and the mart databases.
(iii) The third tier consists of the query interfaces which utilise the API to present the possible BioMart queries and results:
MartView, a web browser interface.
MartService, a web services interface.
MartURLAccess, a URL based access to MartView.
In this section we describe how to use the MartView web interface with an example, followed by a few more biologically relevant queries that users may perform with the currently available mart interfaces. For the first example, we show how to retrieve "the Ensembl mouse genes and genomic locations in the first 10 Mbp of chromosome 1 region that have already been targeted as part of the International Mouse Mutagenesis Consortium ".
The user now has the genomic locations of all the mouse genes in the first 10 Mbp of chromosome 1 but not whether the genes have been targeted yet in the mouse knockout project. To finish this example, a second dataset needs to be added to the query, this time retrieving mouse knockout data from the BioMart located at the Sanger Institute, UK. The second (lower) Dataset bar on the left hand pane has to be clicked and the Gene targeting dataset from the HIGH THROUGHPUT GENE TARGETING AND TRAPPING database selected. The status filter is then set to 'ES cells - Targeting confirmed' and the attributes set to Gene symbol, EUCOMM, KOMP, NorCOMM (component projects of the International Mouse Mutagenesis Consortium) and status. This time, when the results button is clicked, a single mouse gene (St18) in the chromosome 1 region is shown to be have reached the "targeting confirmed stage" and has been assigned to the KOMP project (Figure 1E).
Another common use of BioMart is the analysis of the genes up-regulated in a particular microarray experiment. For example, a user might retrieve "1 kb of upstream sequences from a cluster of human genes identified by an expression profile experiment on an Affymetrix Genechip U95Av2".
BioMart may also be used for annotating experiments or for mapping identifiers to genes and vice a versa. For instance, instead of exporting sequence in the above example, the user could have chosen gene identifiers and names mapping to the uploaded microarray probe IDs. Ensembl contains a wide range of external identifiers for its genes making detailed annotation possible e.g. GO (Gene Ontology), EMBL (European Molecular Biology Laboratory)/Genbank, UniProt (Universal Protein Resource), UniGene, Pfam (Protein Family), PDB (Protein Data Bank) and RefSeq identifiers as well as official names from the naming committees of each species, such as the HGNC (HUGO Gene Nomenclature Committee) and MGI (Mouse Genome Informatics) symbols.
Recently, many other groups have applied the BioMart technology to help their scientists answer complex queries such as those shown above. For example, the CASIMIR (Coordination and Sustainability of International Mouse Informatics Resources) consortium has created a mouse portal prototype .
The Perl API (for download and install instructions see ) is self-explanatory with the help of an example. The best way to learn and, indeed, generate the API scripts is to use the Perl button in the top pane of any MartView site after a manual query has been defined. The script below extracts the mouse and human Ensembl Gene ID and genomic positions for all human genes on chromosome 1 that have a mouse orthologue on chromosome 2. Just like the website, generating a query involves setting a dataset, adding filters and attributes and selecting an output format. The user can even get a count of the results just like the website.
my $confFile = "PATH TO YOUR REGISTRY FILE UNDER biomart-perl/conf/"
my $initializer = BioMart::Initializer-> new('registryFile'=>$confFile, 'action'=>'cached');
my $registry = $initializer->getRegistry;
my $query = BioMart::Query-> new('registry'=>$registry,'virtualSchemaName'=>'default');
my $query_runner = BioMart::QueryRunner->new();
# to obtain count
# print $query_runner->getCount();
# to obtain unique rows only
MartServices, BioMart's RESTful type web services, is available as part of the MartView web application and, as with all the BioMart interfaces, designed to be as simple to use as possible. It is available from the same location as MartView, i.e. if the user accesses MartView using http://www.myurl.org/biomart/martview then they would access the MartServices using http://www.myurl.org/biomart/martservice. Both overview information (metadata) about the services and queries are submittable (for details on the metadata services see the documentation on the main site).
Like the Perl API, the query XML for MartServices is self-explanatory and again the best way to learn and, indeed, generate it is to use the XML button in the top pane of any MartView site to produce a representation of the currently configured query. The XML to recreate the Perl API example above (extracting the mouse and human Ensembl Gene ID and genomic positions for all human genes on chromosome 1 that have a mouse orthologue on chromosome 2) is shown below:
<?xml version="1.0" encoding="UTF-8"?>
<Query virtualSchemaName = "default" formatter = "TSV" header = "0" uniqueRows = "0" count = "" datasetConfigVersion = "0.6" >
<Dataset name = "hsapiens_gene_ensembl" interface = "default" >
<Filter name = "chromosome_name" value = "1" />
<Attribute name = "ensembl_gene_id" />
<Attribute name = "chromosome_name" />
<Attribute name = "start_position" />
<Attribute name = "end_position" />
<Dataset name = "mmusculus_gene_ensembl" interface = "default" >
<Filter name = "chromosome_name" value = "2" />
<Attribute name = "ensembl_gene_id" />
<Attribute name = "chromosome_name" />
<Attribute name = "start_position" />
<Attribute name = "end_position" />
Again, note how the query is formed by adding Datasets within a Query Tag and filters and attributes within the Datasets. As for the Perl API the output format can be changed with a formatter setting, counts can be performed by setting count="1" and unique rows by setting uniqueRows="1" on the Query. To submit a query to MartServices, this XML has to be posted to http://www.biomart.org/biomart/martservice by appending a query parameter. For example using wget: wget - O results.txt 'http://www.biomart.org/biomart/martservice?query=MY_XML' replacing MY_XML with the XML obtained above.
The MartView web interface can be pre-populated with an existing query using URL/XML requests. This can be achieved by sending the XML query as described in the previous section to the following URL
Equivalently, an XML free representation of the same query can be sent to the following URL for similar results
<XML_REQUEST> can be replaced with the XML query as described above for MartServices and <URL_REQUEST> represents the same query in a URL format. As for the Perl API and MartService interfaces, construction of the URL request is best handled by building the query using the MartView website and then using the URL button in the top pane of MartView. The URL/XML request feature is obviously useful for bookmarking favourite queries as well as for constructing canned queries when linking from an external site directly to MartView.
A number of external software packages have incorporated BioMart querying capabilities into their systems. Generally, this has been carried out to improve their software by importing data through BioMart into their system for: (i) further analysis using the existing services they provide (Galaxy, BioConductor, Taverna) or (ii) to add further annotation to their results (Cytoscape). This integration has been made possible through MartServices. All the requests generated by these external packages run against the BioMart central portal. Using BioMart through these external packages expands the usefulness of both BioMart and these external tools. Therefore, brief descriptions and examples of this integrated usage are presented below. BioMart has also been improved by the incorporation of external software technologies. BioMart can be easily configured to become a DAS annotation server for viewing of data through various DAS clients.
Galaxy [5, 23] integrates genomic sequences, alignments and functional annotation through an interactive and simple web portal, so no installation is required. The system allows users to gather data using resources like BioMart or the UCSC (University of California Santa Cruz) Table Browser. The user can then manipulate the data in a variety of ways, such as simple intersections (e.g. selecting the genes common to two BioMart result sets), unions, and subtractions, or more complex analysis using tools such as those from the EMBOSS (European Molecular Biology Open Software Suite) package . An example of this integrated analysis is shown in Additional file 2.
BioConductor  is open source software for the analysis of genomic data. BioConductor is based on the R programming language that is especially suited for statistical analysis. Comprehensive instructions on how to install R and BioConductor are provided on their site . The biomaRt package provides an API to query BioMart databases for use within BioConductor.
biomaRt mimics the functionality of the Perl API, allowing retrieval of any of the information that the other BioMart interfaces allow. A second set of functions is tailored towards Ensembl and include commonly used queries in microarray data analysis. Using biomaRt, a user can for example annotate the features on an array with official gene names, GO identifiers/descriptions and OMIM (Online Mendelian Inheritance in Man) terms retrieved via identifiers, such as Affymetrix, Locuslink, RefSeq or EntrezGene IDs. The package also provides homology mappings between these identifiers across all the species present in Ensembl.
The first stage in using biomaRt is to load the library and choose a mart to use:
Output from the listMarts command of the BiomaRt library
ENSEMBL 49 GENES (SANGER)
ENSEMBL 49 HOMOLOGY (SANGER)
ENSEMBL 49 PAIRWISE ALIGNMENTS
ENSEMBL 49 MULTIPLE ALIGNMENTS
ENSEMBL 49 VARIATION (SANGER)
Next a dataset is chosen:
ensembl = useMart("ensembl")
Output from the listDatasets command of the BiomaRt library
To set the dataset to be queried, the useMart function is used:
human = useMart("ensembl", dataset="hsapiens_gene_ensembl")
The query is constructed with the getBM function. For example, the following will return the Ensembl Gene ID and genomic position for genes up-regulated in an Affymetric HG U133 Plus 2 experiment:
getBM(attributes=c("ensembl_gene_id","chromosome_name","start_position","end_position"),filter="affy_hg_u133_plus_2",values=c('215984_s_at', '203174_s_at', '215984_s_at'), mart=human)
Output from the getBM command of the BiomaRt library
Cytoscape  is open source software for the visualisation of molecular interaction networks and their integration with other biological data such as gene expression profiles. Cytoscape uses both web services and BioMart MartServices to retrieve this extra annotation (see Additional file 3).
The Taverna workbench [7, 26], another open source software package which integrates BioMart, allows interoperation between local and remote analysis tools and databases by providing an environment for the design and execution of workflow experiments. Taverna is able to utilise BioMart, web services and BioMoby  services allowing its users to combine over 3000 different resources and analysis tools, providing a flexible and extensible platform for bioinformatics research. Overall, Taverna allows bioinformaticians to build automated protocols that access each data source and integrate the collected results into a suitable form for biologists to explore (see Additional file 4).
Any BioMart server can be easily configured to act as a DAS annotation server  so that any DAS client such as GBrowse or the Ensembl genome browser can display the data stored in BioMart. DAS offers a simple system for data federation where a DAS client may be used to view the data from several sources in a single, usually graphical, interface. For example, genes stored in a BioMart dataset, such as those affected in a mouse strain stored in the European Mouse Mutant Archive (EMMA) repository may be displayed as tracks in Ensembl contigView along with the usual gene tracks (see Additional file 5).
A list of the sources available from our central portal is available . This server currently returns annotation across a 'segment'. Possible 'segment' values could be a feature identifier or a genomic region defined by chromosome:start,end whereby start and end are optional. For example, an ensembl Homo sapiens DAS data source called 'default__hsapiens_gene_ensembl__ensembl_das_gene' can be accessed as:
An important feature we intend to implement in the near future is secure data access. This is vital where certain data is sensitive and adding this feature will make BioMart an even more attractive solution for organisations looking for controlled access. Adoption of BioMart will then allow secure and simple browsing of their private data, as well as the power of integrated querying of this data with the available public BioMarts.
We will also focus on new interfaces that further simplify multiple dataset querying. In these new interfaces, once dataset(s) are chosen, all the attributes and filters will be presented to the user as if from a single data source, even if they originate from distributed BioMarts. These interfaces will also address the needs of users who require a simpler, more limited, query tool. In addition, we will expand the options available for viewing, analysing and saving results. For example, we will offer gene reports containing all the information we have on a gene in a nicely formatted web page. We will also offer graphical display of results such as locations on a karyogram or a bar chart display of distributions. Furthermore, statistical analysis of results will be provided e.g. whether particular GO terms are enriched in the result set.
In this paper, we have demonstrated that BioMart provides a range of simple, but powerful, interfaces for querying biological data. These may be used for many important research applications such as the data mining of large genomic resources, identification of candidate disease associated genes and variations within them, and the annotation of genome-wide experiments such as microarray studies. BioMart's architecture allows integrated querying of resources at different locations. As the number of publicly available BioMarts increases, users will be able to ask ever more complex queries. The presence of a services layer (MartService) and Perl API provides easy programmatic access for more technical users to script against BioMart and to integrate our software into their own systems. However, we hope the simplicity of generating the MartService and Perl API queries via the website will encourage the novice user to use these interfaces where appropriate. The integration of BioMart with external software adds a further dimension to its usefulness as a research tool. BioMart is also part of GMOD. This project aims to provide a free set of software for creating and administering a model organism database, including genome visualisation, annotation and literature curation. Tools currently exist to produce a BioMart database from the GFF3 gene structure files commonly used by GMOD and further integration of BioMart and other components of GMOD is planned in the near future.
We hope that this description of BioMart, and the direction the system is heading, will have encouraged users and data deployers to explore BioMart for their own biological query requirements.
Text Project name: BioMart
Project home page: http://www.biomart.org
Operating system(s): Any. Local deployment of BioMart requires the Java Virtual Machine 1.3,1.4 or 1.5, Perl 5.6.0 or higher and either Apache 1.3,1.4 or Apache 2.0 or higher are required
Programming language: Java and Perl
Any restrictions to use by non-academics: None.
We thank the Wellcome Trust, EMBL and the European Commission within its FP6 Programme, under the thematic area "Life sciences, genomics and biotechnology for health", contract number LHSG-CT-2004-512092 for providing funding for the project at the EBI. At the Ontario Institute for Cancer Research work funding is provided by the government of Ontario. We are most grateful to Cara Woodwark for her feedback on this manuscript.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.