<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="js7_job_documentation_v1.1.xsl"
		?>
<description xmlns="http://www.sos-berlin.com/schema/js7_job_documentation_v1.1"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://www.sos-berlin.com/schema/js7_job_documentation_v1.1 http://www.sos-berlin.com/schema/js7_job_documentation_v1.1.xsd">
   <job xmlns:xi="http://www.w3.org/2001/XInclude"
        xmlns:xhtml="http://www.w3.org/1999/xhtml"
        name="SQLPLUSJob"
        title="Start SQL*Plus client and execute a sql*plus script">
        <script language="java" java_class="com.sos.jitl.jobs.db.oracle.SQLPLUSJob"
              resource="1"/>
    </job>
   <configuration>
      <params order="1" id="return_parameter">
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="sql_error"
                required="false"
                default_value="">
                <note>
                    <title>Error Messages of the SQL*Plus Client or the database server</title>
                    <p>
                        If errors occur during execution of SQL statements then the error messages will be stored with the
                        "sql_error" result variable in the order of occurrence.
                        If no errors occur then the value of the return variable is empty.
                    </p>
                </note>
            </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="std_out_output"
                required="false"
                xml:base="file:/usr/scheduler/scheduler.build/workspace/020.release.2.5.sos-components.centos8/sos-jitl/sos-jitl-jobdoc/src/main/resources/includes/com.sos.jitl.jobs.sql/param_std_out_output.xml">
 
            <note>
               <title>Content of stdout</title>
               <p>
        The output of the process to the stdout channel is reported from the value of this variable.
        </p>
            </note>
         </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="exit_code"
                required="false"
                DataType="SOSOptionString"
                xml:base="file:/usr/scheduler/scheduler.build/workspace/020.release.2.5.sos-components.centos8/sos-jitl/sos-jitl-jobdoc/src/main/resources/includes/com.sos.jitl.jobs.sql/param_exit_code.xml">
            <note>
               <p>
If SP2 or Oracle errors occur (SP2-*, ORA-*) then the job is terminated abnormally.
The exit code is set to 98.
        </p>
            </note>
            <note>
               <title>stderr channel not empty</title>
               <p>
If the stderr channnel ist not empty then the exit code is set to 99.
        </p>
            </note>
            <note>
               <title>Job return code</title>
               <p>
        The return code of the process is provided with the value of this return variable.
        The return code is zero if the process complets without errors.
        </p>
            </note>
         </param>
      </params>
      <params id="job_parameter">
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="shell_command"
                required="true"
                default_value="sqlplus">
                <note>
                    <title>Path to the SQL*Plus Client</title>
                    <p>
                        The argument specifies the path to the SQL*Plus Client, frequently "slqplus".
						The SQL statements of the job are processed by the SQL*Plus Client.
					</p>
                </note>
            </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="command_line_options"
                required="false"
                default_value="">
                <note>
                    <title>SQL*Plus Client Options</title>
                    <p>
                        The value of this argument contains common command line options available for the SQL*Plus Client.
                        e.g. -S -L
                        
                    </p>
                </note>
            </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="db_url"
                Category="database"
                DefaultValue=""
                required="true"
                DataType="SOSOptionString"
                xml:base="file:/usr/scheduler/scheduler.build/workspace/020.release.2.5.sos-components.centos8/sos-jitl/sos-jitl-jobdoc/src/main/resources/includes/com.sos.jitl.jobs.sql/param_sqlplus_db_url.xml">
	 
	           <note>
		             <title>Service ID of the Database</title>
		             <p>
               The database connection is specified by the job with the <paramref>db_url</paramref>, <paramref>db_user</paramref> and <paramref>db_password</paramref>
               arguments lke this: db_user/db_password@db_url, for example:
        </p>
               <pre>
               sys/sys_admin@localhost as sysdba
        </pre>
            </note>
    
         </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="db_user"
                required="false">
                <note>
                    <title>The user accocunt for access to the database</title>
                    <p>
                        Database user accounts can be retrieved from a credential store by use of the cs://path/entry@user property.
                    </p>
                </note>
            </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="db_password"
                required="false">
                <note>
                    <title>The password for access to the database</title>
                    <p>
                        Passwords for database accounts can be retrieved from a credential store by use of the cs://path/entry@password property.
                    </p>
                </note>
            </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="command_script_file"
                required="false"
                DataType="SOSOptionCommandString"
                OptionName="SQLPlusScriptFile"
                see=""
                prefix=""
                scope="all"
                xml:base="file:/usr/scheduler/scheduler.build/workspace/020.release.2.5.sos-components.centos8/sos-jitl/sos-jitl-jobdoc/src/main/resources/includes/com.sos.jitl.jobs.sql/param_sqlplus_script_file.xml">

            <note>
               <title>Path to a file that contains the SQL statement(s)</title>
               <p>
      	SQL statements are specified from an input file to the SQL*Plus Client.
      	SQL statements included in a file have to be separated by a semicolon and a subsequent new line.
       </p>
               <p>
         Placeholders are replaced in SQL statements by the job. This is achieved by embedding a variable in the form ${argument}
          in the SQL statement. On execution of the job the variable "argument" is assigned the value of the
          argument with the same name.
       </p>
            </note>
    
            <codeexample type="sql*plus">
               <embeddedExample>
WHENEVER SQLERROR EXIT SQL.SQLCODE
WHENEVER OSERROR EXIT FAILURE

column end_date new_value BCY_DATE
column period new_value PN_YEAR_PERIOD
column period_prev new_value PN_YEAR_PERIOD_PREV

select '0' as end_date from dual;
prompt SET end_date IS &amp;BCY_DATE;
/
select '0' as period from dual;
prompt SET period IS &amp;PN_YEAR_PERIOD;
/
select '0' as period_prev from dual;
prompt SET period_prev IS &amp;PN_YEAR_PERIOD_PREV;
/
prompt SET end_date IS 1;
exit;
</embeddedExample>
            </codeexample>

            <codeexample type="xml">
               <embeddedExample>

&lt;param  name="command_script_file" value="
WHENEVER SQLERROR EXIT SQL.SQLCODE
WHENEVER OSERROR EXIT FAILURE

column end_date new_value BCY_DATE
column period new_value PN_YEAR_PERIOD
column period_prev new_value PN_YEAR_PERIOD_PREV

select '0' as end_date from dual;
prompt SET end_date IS &amp;BCY_DATE;
/
select '0' as period from dual;
prompt SET period IS &amp;PN_YEAR_PERIOD;
/
select '0' as period_prev from dual;
prompt SET period_prev IS &amp;amp;PN_YEAR_PERIOD_PREV;
/
prompt SET end_date IS &amp;BCY_DATE;
exit;
"/&gt;

</embeddedExample>
 
               <explanation>
                  <title>Multi-line argument values</title>
                  <p>
An argument value can include line breaks.
</p>
               </explanation>
            </codeexample>

            <codeexample type="xml">
               <embeddedExample>
        ...
        &lt;param  name="command_script_file" value="./config/scripts/sqlplus/sqlplus2.sql"/&gt;
        ...
</embeddedExample>
 
               <explanation>
                  <title>Parameter value from file</title>
                  <p>
</p>
               </explanation>
            </codeexample>

         </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="include_files"
                required="false"
                default_value="">
                <note>
                    <title>SQL Statements to include from files</title>
                    <p>
                    	This argument can be specified to embed additional SQL statements from files.
                    	The SQL*Plus script is prepended the specified files.
                        A number of files can be specified separated by a semicolon.
                    </p>
                </note>
            </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="command"
                required="false"
                default_value="">
                <note>
                    <title>Command to be executed by the SQL*Plus Client</title>
                    <p>
                        The argument specifies a command to be prepended the SQL script file specified
                        by the <paramref>command_script_file</paramref> argument and
                        after optional files specified by the <paramref>include_files</paramref> argument are embedded.
					</p>
                </note>
            </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="timeout"
                required="false"
                data_type="integer">
                <note>
                    <title>Timeout for the command to be executed by the SQL*Plus Client</title>
                    <p>
                        The argument specifies the timeout in seconds for the command to be executed by the SQL*Plus Client. The default is unlimitted.
					</p>
                </note>
            </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="variable_parser_reg_expr"
                default_value="^SET\s+([^\s]+)\s*IS\s+(.*)$"
                required="false"
                DataType="SOSOptionRegExp"
                Alias="VariableParserRegExpr"
                xml:base="file:/usr/scheduler/scheduler.build/workspace/020.release.2.5.sos-components.centos8/sos-jitl/sos-jitl-jobdoc/src/main/resources/includes/com.sos.jitl.jobs.sql/param_variable_parser_reg_expr.xml">

 
            <note>
		             <title>Regular Expression for Parameter parsing</title>
               <p>The process output is parsed using this regular expression. Any matches are available as order variables.
          </p>
               <ul>
                  <li>The regular expression has to include two groups.</li>
                  <li>The first group states the variable name and the second group states the value.</li>
               </ul>
               <p>
          The 'prompt' in SQL*Plus writes the output. Example:
          </p>
               <pre>
            WHENEVER SQLERROR EXIT SQL.SQLCODE
            WHENEVER OSERROR EXIT FAILURE
            prompt ready;
            prompt foo = bar;
            prompt SET hello IS world;
            exit;
               </pre>
               <p>
          The 'dbms_output.put_line()' function in PL/SQL writes the output. Example:
          </p>
               <pre>
            begin
                dbms_output.put_line('ready');
                dbms_output.put_line('Set hello IS world');
                dbms_output.put_line('foo = bar');
            end;
               </pre>
               <ul>
                  <li>variable_parser_reg_expr="^SET\s+([^\s]+)\s*IS\s+(.*)$" returns the order variable assignment "hello=world"</li>
                  <li>variable_parser_reg_expr="^([^=]+)\s*=\s+(.*)$" returns the order variable assignment "foo=bar"</li>
               </ul>

            </note>
            <note xml:base="/usr/scheduler/scheduler.build/workspace/020.release.2.5.sos-components.centos8/sos-jitl/sos-jitl-jobdoc/src/main/resources/includes/common/notes/note_regexp_remarks.xml">
               <p>
            Some remarks on regular expressions used in JobScheduler and YADE.
        </p>
               <p>
            A regular expression is not a
            <a href="http://en.wikipedia.org/wiki/Wildcard_character#Files_and_directories">wildcard</a>.
            To get an idea of the differences users can check the meaning of the wildcard *.txt, which will
            select all file names with the file name extension ".txt".
            A regular expression works in a similar way, the "wildcard" just looks like "^.*\.txt$".
            This looks a little bit strange but it is more flexible and powerfull for filtering file names
            than the concept of wildcards, if users want to filter more complex names or patterns.
        </p>
               <p>
            The general syntax of a
            <a href="http://en.wikipedia.org/wiki/Regular_expression">regular expression</a>, also referred to as regex or regexp,
            is described
            <a href="http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html">
            here</a>.
            It is different to other regular expression implementations, for example in Perl.
        </p>
               <p>
            For further information about regular expressions see
            <a href="https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html">java.util.regex.Pattern</a>
               </p>
               <p>
            If users want only parts of a regular expresion to be case sensitive, then they have the option to use
            the (?-i) and [optionally] (?i) modifiers: (?-i)G[a-b](?i).*
        </p>
               <p>
            Users dealing with Unicode characters should check if their regular expression engine properly supports Unicode.
        </p>
            </note>
            <note xml:base="/usr/scheduler/scheduler.build/workspace/020.release.2.5.sos-components.centos8/sos-jitl/sos-jitl-jobdoc/src/main/resources/includes/common/notes/note_regexp_additional.xml">
		             <p>
            As an extension to regular expressions the [date:] placeholder is supported. When Using this feature it is possible to create a regular expression which
            depends on the date that is specified with the date format string.
	    </p>
	    
            </note>
         </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="ignore_ora_messages"
                required="false"
                default_value=""
                DataType="SOSOptionStringValueList"
                Category="sql,pl/sql,sql*plus">
                <note>
                    <title>Ignore "ORA-" Errors</title>
                    <p>
                        The value of this argument is a list of identifiers for Oracle server error messages that should be ignored.
                        Multipe identifiers have to be separated by a semicolon (;).
                        The prefix "ORA-" can be omitted.
                    </p>
                </note>
            </param>
         <param xmlns:xi="http://www.w3.org/2001/XInclude"
                xmlns:xhtml="http://www.w3.org/1999/xhtml"
                name="ignore_sp2_messages"
                required="false"
                default_value="">
                <note>
                    <title>Ignore SQL*Plus Errors</title>
                    <p>
                        The value of this argument is alist of identifiers for Oracle SQL*Plus error messages that should be ignored.
                        Multipe identifiers have to be separated by a semicolon (;).
                        The prefix "SP2-" can be omitted.
                    </p>
                </note>
            </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="credential_store_file"
                required="false"
                group="Credential store parameters">
            <note>
        	      <title>Location of the Credential Store database file</title>
               <p>
                The path to the KeepPass database file that contains the credentials for access to the database.
            </p>
               <p>
                Alternatively it is possible to specify the value in the URL with the <code>?file=</code> query parameter.
            </p>
            </note>
         </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="credential_store_key_file"
                required="false">
            <note>
               <title>Location of the Credential Store Key File</title>
               <p>
                The path to the KeePass key file that represents the key for access to the credential store.
                This requires a key file to be assigned the KeePass database file. If no KeePass key file is specified, 
                then the file name is assumed from the basename of the KeePass database file and the extension .key.
            </p>
               <p>
                Alternatively it is possible to specify the value in the URL with <code>?keyfile=</code> query parameter.
            </p>
            </note>
         </param>
         <param xmlns:xhtml="http://www.w3.org/1999/xhtml"
                xmlns:xi="http://www.w3.org/2001/XInclude"
                name="credential_store_entry_path"
                required="false">
            <note>
        	      <title>Path to entries in the Credential Store</title>
               <p>
                The argument is used to specify a common path that is added to all entries.<br/>
                Example: <br/>
                credential_store_entry_path=/sos/jobscheduler<br/>
                password=cs://mydb@password<br/>
                In the KeePass database the entry is found in /sos/jobscheduler/mydb@password.
            </p>
               <p>
                Alternatively it is possible to specify the value in the URL with the <code>?entry_path=</code> query parameter.
            </p>            
            </note>
         </param>
      </params>
   </configuration>
   <documentation xmlns:xi="http://www.w3.org/2001/XInclude"
                  xmlns:xhtml="http://www.w3.org/1999/xhtml">
        <note>
            <title>What is the purpose of this job?</title>
            <p>
	        	The job executes a SQL*Plus script.
	            The job runs the SQL*Plus Client in a separate process.
	            It keeps the job in control of the program flow.
	        </p>
        </note>
        <note>
            <title>How is the SQL script defined?</title>
            <p>
                SQL statements are stored to a separate file that is specified by the <paramref>command_script_file</paramref> argument.
	        </p>
            <p>
				Before the SQL*Plus Client is launched for execution of the SQL script, the following steps are applied
				First, for each parameter a "DEFINE" directive is generated.
				These directives are written to a temporary file.
				Thereafter, the specified additional files (see <paramref>include_files</paramref>) are copied to the
				temporary file.
                Then the original SQL script file is added to the temporary file.
				Subsequently, all variables specified directly with SQL statements are substituted by arguments values.
				Finally the temporary file is used for execution with the SQL*Plus Client.
			</p>
        </note>
        <note>
            <title>How is it possible to use parameters in SQL scripts?</title>
            <p>
	     		Order variables can be used in the SQL script in two ways:
            </p>
            <p>
	     		First, all arguments can be used in SQL*Plus statements with their name being preceeded with an ampersand "&amp;".
	     		The arguments are defined as SQL*Plus variables by the job.
	     		The job creates a "DEFINE" directive for each argument.
	     	</p>
            <p>
	     		Second, the names of arguments can be used directly in the SQL script.
	     		The names are replaced by value of the respective argument.
	     	</p>
        </note>
        <note>
            <title>How do I access stdout and stderr channels?</title>
            <p>
	        	The contents of the stdout and stderr channels are available from values of the corresponding result variables.
	     		Any subsequent jobs and instructions in a workflow can access the result variables.
			</p>
        </note>
        <note>
            <title>How can results be reused in subsequent jobs?</title>
            <p>
                To report return values the job adds result variables.
                The first set of variables is static in the sense that variables are always provided, such as the
                variables explained with section "return variables".
                In addition, order variables can be added dynamically.
                The job parses output of the process and extracts variable names and values using a regular expression.
                The regular expression is defined with the <paramref>variable_parser_reg_exp</paramref> argument.
            </p>
            <p>
                Result variables are available to jobs and instructions in subsequent nodes of a workflow.
                The values of resul variables can be modified by subsequent jobs and instructions.
            </p>
        </note>
        <note>
            <title>How are errors detected and handled?</title>
            <p>
	        	The job detects errors reported by the SQL*Plus Client.
	     		Such errors use an identifier starting with the string "SP2-" followed by a number.
	     	</p>
            <p>
	     		In addition, the job detects errors reported by the database server on execution of the SQL script.
	     		Such errors use an identifier starting with the string "ORA-" followed by a number.
	     	</p>
            <p>
	     		If SP2-* or ORA-* errors occur then the job is terminated abnormally.
	     		A return code 98 is set.
	     	</p>
            <p>
	     		In addition, the error message is reported as the value of the <paramref>sql_error</paramref> result variable.
	     	</p>
        </note>
        <note>
            <title>Can errors be ignored?</title>
            <p>
				It is possible to ignore errors.
	  			The <paramref>ignore_sp2_messages</paramref> and <paramref>ignore_ora_messages</paramref> arguments can be used to this purpose.
				The identifiers of the respective errors to be ignored are specified as the values of the arguments.
	    		Error messages are available with the stdout channel, even if they are ignored.
	    	</p>
        </note>
        <p>
            <codeexample type="sql*plus">
                <embeddedExample type="sql*plus">WHENEVER SQLERROR EXIT SQL.SQLCODE
WHENEVER OSERROR EXIT FAILURE

column end_date new_value BCY_DATE
column period new_value PN_YEAR_PERIOD
column period_prev new_value PN_YEAR_PERIOD_PREV

select '0' as end_date from dual;
prompt SET end_date IS &amp;BCY_DATE;
/
select '0' as period from dual;
prompt SET period IS &amp;PN_YEAR_PERIOD;
/
select '0' as period_prev from dual;
prompt SET period_prev IS &amp;PN_YEAR_PERIOD_PREV;
/
prompt SET end_date IS 1;
exit;
</embeddedExample>
                <explanation>
                    <title>Example: SQL*Plus Script</title>
                    <p>
	            		The following variables are returned,
	              		see the <paramref>variable_parser_reg_expr</paramref> agument.
	              		To this purpose, the output of the script is parsed with the regular expression "^SET\s+([^\s]+)\s*IS\s+(.*)$".
	            	</p>
                    <p>
	             		In the above example the following result variables are returned:
	            	</p>
                    <p>
                        <ul>
                            <li>end_date=1</li>
                            <li>period=0</li>
                            <li>period_prev=0</li>
                        </ul>
                    </p>
                    <p>
                        If a result variable is set a number of times then the last assignment is used.
                    </p>
                </explanation>
            </codeexample>
        </p>
    </documentation>
</description>