MINIMIZE(1l)	  MISC.	REFERENCE MANUAL PAGES	     MINIMIZE(1l)



NAME
     minimize -	minimize CDIF tagging and  rearrange  problematic
     elements

SYNOPSIS
     minimize [-cEedhlmnrs] [[-fz] -o suffix]  [-v  cdif_version]
     [CDIF_file...]

DESCRIPTION
     minimize takes as its input one or	more valid CDIF	files, or
     a	valid  CDIF  text  from	 standard input, and sends to its
     standard output, or to new	files,	reformatted  versions  of
     its  input	 with  minimized  tagging (that	is, with end tags
     supplied even where the DTD says that they	may be	omitted).
     Minimization  is achieved by passing the input files through
     the sgmls program,	and post-processing  the  output  to  re-
     create a valid CDIF document.

     Optionally, certain ``problematic'' elements  are	moved  to
     positions where they are not likely to cause problems during
     during subsequent marking of segments.   Such  elements  are
     those which may themselves	contain	one or more segments, but
     which are allowed by the DTD to occur at positions	which may
     turn  out to be in	the middle of a	segment.  The set of such
     elements  currently  consists  of	<caption>,   <note>   and
     <stage>.  An empty	<ptr> tag is left at the original site of
     each moved	element.  The t	attribute of the  <ptr>	 has  the
     same  value  as the id attribute of the moved element, which
     is	repositioned at	a ``safe'' point, such as after	 the  end
     of	the paragraph origionally containing the element.

     The program uses a	conservative heuristic to decide  whether
     to	 move  an  element:  if	a ``problematic'' element appears
     directly inside an	element	which may not itself contain seg-
     ments  (except  as	 elements of subordinate elements), it is
     not moved.	 If it appears before any  other  content  (apart
     from white-space not considered to	be content) in an element
     which may contain segments, it is not moved.  If it  appears
     anywhere  else - including	inside an element which	is itself
     being moved - it is moved.	 The program is	also conservative
     about  where  to  position	moved elements:	typical	sites are
     after the end of a	<p>, or	before the end of a <divn>.

     Each text is reformatted so as to contain no blank	lines, no
     runs  of  spaces,	and  no	 leading or trailing space on any
     line.  Optionally,	entity references can be modified so that
     they  are always terminated by a semi-colon, even where SGML
     syntax does not require an	explicit terminator; or	 so  that
     they are only terminated by a semi-colon where SGML requires
     an	explicit terminator.  Blocks of	text are not  re-wrapped,
     however.	Rules  stating which start and end tags	should be
     preceeded or followed by a	new-line are also applied.



Sun Release 4.Last change: TGCW41: 2 February, 1993		1






MINIMIZE(1l)	  MISC.	REFERENCE MANUAL PAGES	     MINIMIZE(1l)



     Under the -r option, an attempt is	made to	eliminate as many
     <hi>...</hi> pairs	as possible by assigning to the	attribute
     list of a parent element the r (rendition)	attribute of  any
     <hi>  element  which is the only content of that parent ele-
     ment.  Thus, for example,

	  <p><hi r=bo>Some text.</hi></p>

     would become

	  <p r=bo>Some text.</p>

     Apparently	un-highlighted punctuation is also folded.  Thus


	  <p>&bquo;<hi r=it>He's got it	coming to him</hi>!&equo;</p>

     becomes

	  <p r=it>&bquo;He's got it coming to him!&equo;</p>

     This is a consequence of an apparent inability on	the  part
     of	OCR systems to recognise highlighted punctuation.

     In	a similar manner, the -l option	results	in the	rewriting
     of	 <label>s outside <list>s, which otherwise cause problems
     when segmentation is applied.  For	example,


	  Labels in running text are <label>a)</label>rare;
	  and<label>b)</label>problematic.

     becomes


	  Labels in running text are <omit cause=label desc='a)'>rare;
	  and <omit cause=label	desc='b)'>problematic.

     Under the -s option, the n	attributes of <s> elements can be
     optionally	 be rewritten so as to form a monotonic	ascending
     sequence.

     No	output is produced for a given input file  if  that  file
     does  not	parse  against	the  DTD specified in its DOCTYPE
     declaration, or, if the -v	option is in force,  against  the
     DTD  specified  by	that option.  If the -c	or -h options are
     in	force, no output is produced  if,  respectively,  non-ISO
     646 IRV characters, line-start or line-end	hyphens	are found
     in	an input file.

     Where output is successfully written to a file (rather  than
     to	 standard  output),  the  -z option allows the processing



Sun Release 4.Last change: TGCW41: 2 February, 1993		2






MINIMIZE(1l)	  MISC.	REFERENCE MANUAL PAGES	     MINIMIZE(1l)



     carried out to be recorded	in a corresponding  documentation
     file.

OPTIONS
     -c	  Check	for illegal characters in document  content.   In
	  general,  this  means	 characters  which  are	 legal in
	  ASCII, but not in ISO	646 IRV.   See	TGCW25	for  more
	  information.

     -d	  Do not use  the  CDIF	 SGML  declaration  when  parsing
	  files.  The effect of	this is	to allow files to contain
	  characters which would otherwise be illegal.	(See also
	  -c above.)

     -E	  Ensure that all entity references are	terminated  by	a
	  semi-colon,  even where SGML syntax does not require an
	  explicit terminator.

     -e	  Arrange that entity  references  are	terminated  by	a
	  semi-colon  only where SGML requires an explicit termi-
	  nator.

     -f	  Used in conjunction with the -o option, allows existing
	  files	 to  be	overwritten by the program's output, pro-
	  vided	that those files have write permission.	 Use with
	  caution!

     -h	  Check	for line-start and  line-end  hyphenation  -  the
	  first	 or  final character on	a line,	apart from spaces
	  and start- or	end-tags, is a hyphen.

     -l	  Rewrite <label>s which are not immediately  subordinate
	  to	 <list>	   elements    as    <omit    cause=label
	  desc=content_of_label>.    This    transformation    is
	  required as <label>s,	whilst acceptable outside <list>s
	  in unsegmented texts,	cause problems once  segmentation
	  is applied.

     -m	  Enable the moving of	``problematic''	 elements  -  see
	  above.  -n Produce no	output other than error	messages.

     -osuffix
	  Rather than sending output to	standard output, send  it
	  to  files in the current directory with names	generated
	  by appending suffix  to  the	basenames  of  the  input
	  files.   Existing  files will	not be overwritten unless
	  the -f  option  (see	above)	is  also  in  force.   To
	  overwrite input files	in place, use

	  minimize -f -o '' ...

	  (An argument must be supplied	for -o.)



Sun Release 4.Last change: TGCW41: 2 February, 1993		3






MINIMIZE(1l)	  MISC.	REFERENCE MANUAL PAGES	     MINIMIZE(1l)



     -r	  Where	a <hi> element is the sole content of some parent
	  element  (apart  from	trailing punctuation, and leading
	  and trailing whitespace  and	normalized  quote-marks),
	  fold the attributes of the <hi> element into the attri-
	  bute list of the parent element, and	delete	the  <hi>
	  and </hi> tags.

     -s	  Rewrite the n	attributes of <s> (segment)  elements  so
	  that	they form a monotonic ascending	sequence of five-
	  digit, zero-padded numbers.  This option may be used to
	  regularize  texts  into  which  <s>  elements	have been
	  inserted, or from which they have been  deleted,  after
	  Lancaster's initial automatic	numbering process.

     -vversion
	  Check	input files against CDIF version version,  rather
	  than	the  version  specified	in their DOCTYPE declara-
	  tions.  The output DOCTYPE  declarations  specify  CDIF
	  version version.

     -z	  Valid	only when the -o option	is also	specified, the -z
	  option   results   in	  the	documentation  (Z_)  file
	  corresponding	to each	input file being updated  with	a
	  line	giving	the date, the login name of the	user run-
	  ning minimize, and the command line used.  The line  is
	  inserted,  if	 possible, as the third	line in	the file,
	  or as	the last line if the  file  contains  fewer  than
	  three	 lines.	  If  the  file	 does  not  exist,  it is
	  created.

DIAGNOSTICS
     Various error conditions  associated  with	 non-existent  or
     unreadable	 input	files, and with	input files which are not
     valid CDIF	documents, result in ``file skipped'' messages on
     standard  error.	Error messages produced	by the underlying
     sgmls application in response to invalid CDIF documents also
     appear here.  No output to	standard output, or, under the -f
     option, to	any output file, results from any input	file that
     presents a	problem.

     Under the -z option, warnings are given on	standard error if
     the  documentation	 (Z_)  file  does  not	exist,	cannot be
     created, or is not	writable.  The main output file	is  still
     written under these circumstances.

     More serious error	conditions result in  immediate	 termina-
     tion with a diagnostic message.

     The return	status of the program is zero only if no  warning
     or	error conditions were encountered.





Sun Release 4.Last change: TGCW41: 2 February, 1993		4






MINIMIZE(1l)	  MISC.	REFERENCE MANUAL PAGES	     MINIMIZE(1l)



ENVIRONMENT
     SGML_PATH
	  Used by sgmls.  See the man page for that program.

FILES
     /usr/local/bin/minimize
	  The program itself.  It contains both	this manual  page
	  (use	nroff  -man /usr/local/bin/minimize or similar to
	  print	it), and the tables used to define  the	 elements
	  which	 should	be moved, the points to	which they should
	  be moved, and	the use	of line-feeds  before  and  after
	  start- and end-tags.

AUTHOR
     Dominic Dunlop

SEE ALSO
     perl(1), sgmls(1),	TGCW25:	Markup for non-ISO 646	invariant
     part characters, TGCW30: Corpus Document Interchange Format,
     v1.2

BUGS
     Some ``problematic'' elements, such as those appearing after
     the  last	sentence in a paragraph, are moved unnecessarily.
     This is a result both of the fact	that  sentences	 are  not
     tagged  in	source documents, so that the SGML processor can-
     not know where they end; and  of  end-tag	omission  in  the
     source  document, which means that	the SGML processor cannot
     know the paragraph	has ended  until  it  encounters  another
     start-tag.

     The program only knows about ``safe'' places for problematic
     elements  in  unsegmented	text: if the -m	option is used in
     conjunction with a	 segmented  text  containing  previously-
     moved  elements,  the  elements  will  be moved again.  This
     results in	SGML errors when the resulting file is parsed.

     The program attempts to pass  any	DOCTYPE	 declaration  and
     declaration  subset unchanged (except for the effect of -v	-
     see above)	from its input to its output.  The heuristic used
     to	 guess where the subset	ends will be fooled if the subset
     contains the sequence ``]>'', with	 or  without  intervening
     whitespace.

     If, under the -o option, an output	file is	discovered not to
     be	 writable  for	whatever  reason,  this	is not discovered
     until after the input file	has been  fully	 processed.   (No
     attempt  is  made to open the output file before this stage,
     as	it could result	in an existing file  being  truncated  to
     zero  length  in the event	that errors occured in processing
     the input.)




Sun Release 4.Last change: TGCW41: 2 February, 1993		5






MINIMIZE(1l)	  MISC.	REFERENCE MANUAL PAGES	     MINIMIZE(1l)



     SGML comments in the source text do not appear in the output
     text.

     The id attributes generated for moved elements are	unique in
     the  individual  text,  but need extension	if they	are to be
     unique across a collection	of texts - a corpus.

     Unless the	-d option is in	force, the -c option is	 ineffec-
     tive if the input file contains characters	defined	as unused
     by	the SGML declaration.  Such characters cause SGML errors,
     which  terminate  the processing of a file	before the checks
     enabled by	the -c option can be applied.

     If	fed with input which is	has already minimized,	the  pro-
     gram spends a long	time producing output which is identical,
     save that the order of the	attributes for elements	given  an
     id	attribute by the initial minimization may be changed, and
     that the odd newline may be excised.  After  three	 or  more
     passes, the output	is absolutely identical	to the input.

     Despite a number of tweaks	to speed  it  up,  the	algorithm
     which  attempts  to  fold	the attributes of nested elements
     into the attribute	list of	their parents is quite expensive.
































Sun Release 4.Last change: TGCW41: 2 February, 1993		6