About the Project
The ability to automatically generate fluent, accurate and sufficiently detailed (relative to a given context) descriptions of visual content would dramatically improve access to visual information for blind and partially sighted people (BPS). While image description generation in the general case (where a correct and complete description can be generated for any input image) is far beyond the current state of the art, it has over recent years become feasible for highly controlled, well‐defined subclasses of images.
This PhD project is aimed at the area just beyond the state of the art, going beyond highly restrictive subclasses and focusing on a family of more ambitious, related subclasses, namely line diagrams. These are defined to a first approximation as line drawings composed of geometrical shapes. The project will look at three diagram classes characterised by increasing levels of complexity. In addition to bar chart diagrams, for which description generation is relatively feasible, we have identified two more complex classes of diagrams in which the University of Brighton has internationally leading expertise: Euler diagrams, representing sets and their relationships by means of circles and labels, and line drawings of simple architectural components.
The aim of the PhD is to develop a comprehensive methodology for automatically generating verbal descriptions of diagrams given an abstract representation of diagram structure. The approach will have a generic component (applicable to all types of diagram generation) and a task‐specific component (applicable to a given subclass of diagrams). Determining what kind of description is appropriate in what kind of context (such as type of user, desired level of abstraction or size of diagram) will be very much part of the research. There are several natural language generation (NLG) tools available that can be used as a starting point for this project. It is likely that a probabilistic approach will be desirable, in order to provide robustness and automatic adaptability.
In summary, the objectives of the PhD are:
1. to develop generic probabilistic NLG methods for generating descriptions of diagrams from abstract representations of diagram structure
2. to develop specialised extensions of the methods in 1 above for subclasses of diagrams of increasing complexity:
a. bar charts
b. set diagram
c. simple architectural diagrams.
3. to create datasets of paired diagram images, abstract representations and descriptions for developing and training the NLG methods, and for public release as an important data resource for other researchers
4. to investigate the feasibility of automatically mapping images to abstract diagram representations that can serve as inputs to the NLG module, using existing automatic image analysis and virtualisation techniques
5. to create comprehensive evaluation strategies to facilitate thorough evaluation of the developed methods, both by automatic means and involving BPS participants
6. to recruit user groups of BPS people to contribute to method development and user evaluation.
Funding Status:
This studentship is worth at least £60,300 over 3 years, subject to satisfactory progress.
UK students -
For UK students this comprises £4,620 per year (for 3 years) to cover annual tuition fees and a contribution towards living expenses of £15,480 per year (for 3 years).
You must have resided in the UK for three years prior to starting the studentship to receive funding for tuition fees and a stipend
The value of the studentship will be raised to take into account any rise in annual tuition fees.
EU students -
If you are an EU national and have not resided in the UK for three years prior to the start of the studentship you would not be eligible for a stipend, so you would need to have an alternative source of funding for your living costs. For full details on the residence requirements visit the EPSRC website.
Candidates from outside the EU are not eligible for this studentship.