Quality Control of Documents – Some Examples

>There are a number of assumptions to this discussion:
> - Knowledge is in well defined documents
> - Documents hold well ordered language akin to "specifications"
> - The person reading the document is going to learn more than 9 new > things
> - All knowledge is held in large documents
> - All knowledge will be expressed in formal language.
>> Each of these assumptions can be falsified in any knowledge area.
>> But I don't think this is the key point.

Have to start somewhere, and there has to be a benefit for doing so. These specifications represent a piece of knowledge that is passed between two organisations with different agendas – one wants useful, reliable and long lived equipment, the other wants to make a profit. The knowledge in the document is highly leveraged – people will work for many years based on what they say, and spend millions or billions of dollars in doing so. Can you think of any other knowledge that is both newly created and so highly leveraged?

>What you are proposing only provides quality control on the document itself, and does not provide any quality control on the elements of knowledge that are contained in the document.
>

I apologise for my poor explanation. The intent is to find all the semantic errors – what the document means. I will give more examples, but please do not assume that because only these examples are given, the system can do nothing else. It is intended to have the complete semantic structure, linked to the document structure. The analysis occurs in the semantic structure. This isn’t what people call "semantic analysis", but is intended to be much deeper.

"The turning radius of the Yard Truck shall not exceed 15 m."

The Yard Truck is a sort of truck, and they have a turning circle (diameter) between 10 and 20 metres (non-articulated). This value is outside the range (the author meant diameter, but said radius). Obviously the system had to have background knowledge to recognise the error, but it needs knowledge to be able to read anyway.

"The construct of a Yard Truck shall ensure the ingress of air-borne dust and sand into the cabin."

This says exactly the opposite of what it should. We are not initially setting out to find this sort of error, as it would need a fitness for purpose analysis – there is a standard mentioned, and it would be possible to make inferences from that. For the moment, we hope this sort of error is rare. If it isn’t, the analysis is going to take much longer.

Correction. A simple model of a comfortable human environment – 15 – 25 degrees C, up to a limit of particulates but with no zone of comfort as there is for temperature or noise, noise level of 20-70 dBA can be set up for any sort of human container, including cabins. Then adding to particulates would be seen as inconsistent, whereas having an air heater or piped music would not.

"An emergency brake interlock system shall be provided to prevent the movement of the vehicle with the brake applied."

There are two brakes – a parking brake and a service brake – which brake is meant? You might argue that the parking brake is set and the service brake is applied, but the vehicle is often moving and the service brake is applied just to slow it. Then it must mean the parking brake. But the parking brake is applied with the engine turned off, so the emergency system wouldn’t work as described. I just slipped into the sort of analysis a person can do, and which our system can’t. However, it can see the ambiguity and point that out, and leave it to people to clean it up.

"The computer shall incorporate a high resolution screen to display information."

The intent is perfectly clear, but it does not say "shall display information at a high resolution", so the Contractor can supply high resolution displays running at a low resolution. Why would they do that – because it is cheaper to construct a low resolution signal. This is an example of a piece of equipment with a discrete spectrum of capacity - it can be used at high resolution or low resolution. Being on your guard against someone else reading it with a different agenda can be extremely tedious – it would help if something pointed out any weaknesses.

"The Contractor shall consider the operational requirements described in the Annex as a component of this specification."

This is an unverifiable mental trace.  The operational requirements are not in the form "shall", so they can be ignored.

"Real-Time" means transactions which shall be concluded in sufficient time to meet overall performance requirements.

This is a low level definition which appeals to satisfaction of overall requirements – it is circular.

I hope you can see from the examples that a large element of the meaning of the document is being checked. Plenty more available.

What "knowledge systems" do you mean that actually have the semantics at their disposal and make inferences about them?

I used this as an example. Imagine what you could do with the complete semantic structure of the document, what it means, as a network – this is what the system can or will do. It can’t flesh out a local area in great detail as a person can do, but it can do things that a person can’t do – it can skitter from page 1 to page 50 along a direct semantic link, and "see" the groups that a person constructed with many sentences.

The largest test document we are using is

www.inteng.com.au/SystemsEngineering/bus.htm

(this is a stripped down version of a specification that Victoria Department of Transport allowed us to use – we don’t do diagrams)

If you can read it through and understand everything at once, I salute you. You might argue that a person "well versed in the art" should be able to understand it all (but then it wouldn’t have any errors), but it ranges across many technologies, and programmers will need to read it, with no more knowledge of a bus than how to get on it.

Response to