Mainframe FOCUS Now and in the Future Joel Starkman, Director of Operations Information Builders Inc., FOCUS Division August, 2008.

Mainframe FOCUS Now and in the Future

Joel StarkmanJoel Starkman, Director of Operations, Director of Operations

Information Builders Inc., FOCUS DivisionInformation Builders Inc., FOCUS Division

August, 2008August, 2008

Mainframe FOCUS, Now and in the FutureOverall Message

FOCUS is alive and well Over 700 mainframe FOCUS customers worldwide Mainframe FOCUS is still a significant portion of IBI annual revenue Staff of 20 supports all FOCUS, and much of WebFOCUS backend Far more sophisticated QA mechanism and extensive tests

Release 7.6 - 20% cpu reduction due to IBM/C compiler FOCUS benefits from all new WebFOCUS backend features FOCLOG built into 7.6 – analyze FOCUS usage, predict upgrade effort

Release 7.7 is coming in 2009 z/OS version only; VM/CMS version is stabilized at 7.6 One source base, easier to support CLI support, Business Views, Case-insensitivity, Unicode, more Newer z/OS versions and the impact of LE New features for FOCUS customers under discussion

Integrate with WebFOCUS more closely Take advantage of zIIP* processor z/Linux version of FOCUS

* zIIP: z9 Integrated Information Processor

FOCUS To WebFOCUSDesign Concepts

Core FOCUS Options

Simultaneous User

Host Lang Interface

CONSOLE Options

Multi-Session Option (MSO)

BATCH Components

JCL

CLIST

PDS

WebFOCUS Options

MRE

Report Painter

Report Library

Visual Discovery

GIS

FOCUS to WebFOCUSCurrent Options

*Concept under discussion at IBI

FOCUS to WebFOCUSCurrent WebFOCUS Components


FOCUS to WebFOCUSCurrent FOCUS Components


FOCUS to WebFOCUSFuse Together*


CONSOLEMode

FOCUS to WebFOCUSNew Components to WebFOCUS*


CONSOLEMode

BATCH Mode

FOCUS to WebFOCUS Concept*

“Today you are FOCUS.Tomorrow you are WebFOCUS”

Existing applications merely run – no changes at all All WebFOCUS tools are then available

to bring the application into the web world Considerations and Challenges

Make WebFOCUS more receptive to FOCUS applications, like CRTFORM in a web browser

Retain performance level Upwardly compatible Provide assistance/roadmap to transition FOCUS-to-WebFOCUS license issues Timeframe for IBI to implement (2009?)


New FOCUS Features for Release 7.7Planned or Under Design

New FOCUS 7.7 FeaturesThe Majors

zIIP ENGINE* Offload cpu cycles to cheaper processor and others, say moving

to zIIP Issues and limitations under evaluation

UNICODE SUPPORT For FOCUS, XFOCUS and interfaces

SET COLLATION Handle case-insensitive data

SET MVERBLAYOUT Control multi-verb layout of sort columns, now displayed as stacked rows

BUSINESS VIEWS Restrict read access to database based on business requirements


New FOCUS 7.7 FeaturesCase Insensitivity

“All Cases Are

Created Equal”

AAA

AAa

aAA

aaA

AaA

aAa

Aaa

aaa

SET COLLATION=

New FOCUS 7.7 FeaturesUNICODE Support (edited excerpt from IBI website)

Unicode is a universal character-encoding standard that assigns a different code to every character and symbol in every spoken and/or written language in the world. Only Unicode ensures that you can retrieve and combine data using any combination of languages. Unicode is required with XML, Java, JavaScript, LDAP and other Web-based technologies.

Systems that use the Unicode standard:• Work in any operating environment or language• Can transport data through different systems without corruption• Provide a single multilingual development environment

The server supports Unicode Transformation Format (UTF) called UTF-8 in ASCII environments. This encoding standard assigns each character a code from one to three bytes long. The codes assigned to characters from SBCS languages (Eastern and Western European) are one or two bytes long, and those assigned to DBCS languages (Asian) are three bytes long.

This standard is compatible with the ASCII format because the first 128 UTF-8 codes have the same one-byte representation as the corresponding ASCII codes. For EBCDIC environments, a transformation format called UTF-E assigns each character a code from one to four bytes long.

Main benefit of Unicode is the ability to have multiple languages (European and Asian) in:• Titles, descriptions, and names in synonyms• Headings and prompts in procedures• Column names, titles and sample data in the Data Management Console• Data for all supported adapters

(logo of the Unicode Consortium)

New FOCUS 7.7 FeaturesMVERBLAYOUT

Stacks aggregation into the same column as the data, like a WebFOCUS Vertical Accordion Report

TABLE FILE EMPLOYEESUM CURR_SAL ED_HRSSUM CURR_SAL ED_HRS BY DEPARTMENTEND

CURR_SAL ED_HRS DEPARTMENT CURR_SAL ED_HRS

$222,284.00 351.00 MIS $108,002.00 231.00

PRODUCTION $114,282.00 120.00

DEPARTMENT CURR_SAL ED_HRS

$222,284.00 351.00

MIS $108,002.00 231.00

PRODUCTION $114,282.00 120.00

SET MVERBLAYOUT=ON

New FOCUS 7.7 Features Business “View” of a Database

A VIEW into the original file – no new data, no separate extraction Reflect the business instead of physical field arrangement Have as many different Business Views as required; Direct via USE BV recognized only for extract (TABLE[F], GRAPH, MATCH, MORE) JOIN a Business View to other files Limit the fields accessible in the main file, without DBA Make OS impose a View via security

MFDMFD

Business ViewOriginal

FILE=ORIG,SUFFIX=any

FILE=anyname,VIEW_OF=ORIG

New FOCUS 7.7 Features Business “View” of a Database

FILE=ORIG, SUFFIX=FOC

SEGNAME=SEG1, …

FIELD=SOCSECNO, …

FIELD=NAME, …

FIELD=SALARY, …

SEGNAME=SEG2, PARENT=SEG1

FIELD=COMPANY, …

FILE=ORIG, SUFFIX=FOC

SEGNAME=SEG1, …

FIELD=SOCSECNO, …

FIELD=NAME, …

FIELD=SALARY, …

SEGNAME=SEG2, PARENT=SEG1

FIELD=COMPANY, …

FILE=BV, VIEW_OF=ORIG

SEGNAME=SEGMENT1

FIELD=FULLNAME,,BELONGS_TO=NAME,$

FIELD=SSN,,BELONGS_TO=SOCSECNO,$

FIELD=COMPANY,,TITLE=‘CO NAME’,$

FILE=BV, VIEW_OF=ORIG

SEGNAME=SEGMENT1

FIELD=FULLNAME,,BELONGS_TO=NAME,$

FIELD=SSN,,BELONGS_TO=SOCSECNO,$

FIELD=COMPANY,,TITLE=‘CO NAME’,$

Rename/rearrange fields without touching original fileEach Business View field relates (maps) to an original fieldTABLE with new field names translates to original names;

• attributes and relationships taken from original fieldNew DEFINEs permitted in Business View mfd

This Business View omits SALARY

SET FOCTRANSFORM = ON

New FOCUS 7.7 Features Other Features

User-defined IN-RANGES-OF: IN-RANGES-OF <value1, label1> … <valuen, labeln>

SET HOLDATTR includes DESCRIPTIONS from Master File in HOLD file No limit on the concatenation of data files on the sink (SU) “IS MISSING” and ”IS-NOT MISSING” with TX fields HOLIDAY file no longer only in ERRORS

FILEDEF or ALLOC anywhere, multiple versions allowed New DBCS-specific subroutines: JPTRANS, DEDIT, DSUBSTR MAINTAIN supports FOCUS TX fields

Conditional styling based on sort value for graphs, pie charts, tables… New DB2 interface using CLI provides access to DB2 Stored Procedures

Master files generated dynamically PATTERN function translates field value to pattern of “A”, ”a”, ”9” and other chars REVERSE function reverses letters in a word: REVERSE becomes ESREVER

ISO standard week numbering in H (date/time) subroutines “IN FILE” syntax with –SET (compares &variable to list of literals stored in file) FOCUS and XFOCUS databases support up to 1024 partitions

Advances in FOCUS Quality Assurance

All combined, about 100,000 tests run every night

Advances in FOCUS Quality AssuranceCustom-Built Testing System Envelope

Catalogs and executes all tests across the entire company Captures results and compares to approved base results

Runs every night on every platform on every different daily gen Mainframe (z/OS and VM), Linux, AS400, Windows, Unix platforms

Web-based interface to view results of thousands of tests Drill down to the code, do intelligent cross-platform QA analysis

Most test-focexecs contain 5+ tests each. Via Dialog Manager loops,

many other tests generate 100’s or 1000’s of test variations Date and datetime combinations spanning centuries Thousands of /format variations everywhere formats can appear New IBI invention to run interactive tests in batch (FOCREPLAY)

QA and Programming staff examine results daily Discrepancies are reported and fixed before next run Overnight coverage via world-wide staff

Advances in FOCUS Quality AssuranceNew Testing Concepts

Random test-generation and test combinations Hits combinations we could never predict

Random execution paths simulate live sessions Different every day, non-predictable results, OK if it finishes cleanly

Real customer applications simulated in-house Parallel runs of random test combinations against prior release

Compare results to each other rather than to approved base results

The FOCUS language has combinations.

How can you possibly test every one of them?

New QA Tools and MechanismsAll created in the last 3 years

WHERE GeneratorSU Stress

Mechanism

SU ScenariosMODIFY, MAINTAIN, HLI

TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests

Intelligent Testing System

Customer Applications

New QA Tools and MechanismsSU Scenarios


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsSU Scenarios

Plays every logical combination (scenario) that forces two or more users to collide on specific records or database paths with conflicting intentions to Include, Update and Delete that record or path

Results in FOCCURRENT return codes that identify the conflict, or predictably affects a TABLE that is processing at the same time

Plays all of these

against each other: MAINTAIN MODIFY TABLE HLI Commit


New QA Tools and MechanismsParser Stress Tests


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsParser Stress Tests

TABLE FILE BY SUM BY/BY_BY AS 'BY'BY BY PRINT BY COMPUTE BY = BY/BY; BY BY BY BY SUBTOTALON BY SUBHEAD "BY <BY " END

Parser

Stress Tests

Use every FOCUS keyword in every possible position in the TABLE syntax to uncover language parser inconsistencies

TABLE FILE PRINT

SUM PRINT/PRINT AS 'PRINT'

PRINT PRINT

COMPUTE PRINT/A10 = PRINT|’PRINT’;

BY PRINT SUBTOTAL

ON PRINT SUBHEAD

"PRINT <PRINT "

END

New QA Tools and MechanismsTABLE Syntax Generator


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsTABLE Syntax Generator

Generates totally random but valid TABLE syntax against any Master Within syntactic rules, randomly chooses keywords, fields, prefixes,

sorts, WHERE/IF criteria, computations, quoted text, summary lines Invents nested IF/THEN/ELSE expressions and/or calculations within

random numbers of DEFINEs, COMPUTEs, RECAPs, WHENs Randomly creates FRL, MORE, Stylesheets, aliases, multi-verbs

TABLE

Syntax Generator

Generates 1000’s of TABLE combinations every day

Compares current test release to any previous release

New QA Tools and MechanismsCustomer Applications


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsCustomer Applications

Customer passes Customer passes application to Information application to Information

BuildersBuilders

Customer environment is Customer environment is duplicated or closely duplicated or closely

simulated on IBI hardwsimulated on IBI hardw

Database is snapshot of Database is snapshot of actual data or contrived actual data or contrived

data valuesdata values

Application is validated on Application is validated on current gen of next release current gen of next release

– bugs fixed– bugs fixed

Customer informed of Customer informed of pass/fail results and pass/fail results and

migration adjustmentsmigration adjustmentscustomer applications running in-house today!

12345678


New QA Tools and MechanismsWHERE Generator


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsWHERE Generator

Generates 1000’s of complex, syntactically valid WHERE combinations Parenthetical expressions, calculations, subroutine calls, random field

selections, full range of comparison operators Dynamically incorporated into standard TABLE template and executed

WHERE Generator

New QA Tools and MechanismsIntelligent Testing System


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsIntelligent Test System

Randomly combines cataloged individual tests into consecutive executions, looking for unpredictable interactions

Runs twice - against current test release, and a stable previous release Compares results, extracts suspicious activity, isolates the causal test,

reruns separately, reports analysis to QA for further investigation Every day, the test environment (SETs, etc.) is altered on the fly to expose

tests to originally-unintended conditions Tests may be physically enhanced automatically for targeted testing

Insert ON TABLE HOLD FORMAT HTML into every report Add MORE syntax to every report to effectively double results


New QA Tools and MechanismsSU Stress Mechanism


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsSU Stress Mechanism

SU Stress Mechanism

Objective: Crash the sink machine or damage a database, and capture the evidence for evaluation

Random number of tasks per user, random task selection based on weighted frequencies, random inter-task and intra-task time delays, random data selection. Conflicts are unpredictable, like real applications.

Randomly performs Includes, Updates, Deletes, TABLE extracts, volume loads, intentional delays, user out to lunch or crash, HLI via COBOL

Settable # of simultaneous users; users drop off – new ones start up Run it for 10 minutes or 10 days Tracks every activity, generates traces, analyzed after run completes

Tens to 100’s of simultaneous users randomly perform a large variety of

tasks against sink machine databases in real time

New QA Tools and MechanismsSystem Stress Tests


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



New QA Tools and MechanismsSystem Stress Tests

Concentrate specifically on how FOCUS interacts with the operating system environment

Expect files to be sized, stored, opened, closed, appended, removed, etc. properly at expected times with expected return codes

Disks, directories, pds’s, temp areas, etc. to be accessible and utilized as expected

FOCUS’s awareness of other environmental conditions and changes that affect its processing, with proper messaging and handling

System Stress Tests

Advances in FOCUS Quality Assurance


Mechanism


TABLE

Syntax Generator

Parser

Stress Tests

System Stress Tests



Mainframe FOCUS, Now and in the Future

Mainframe FOCUS Now and in the Future Joel Starkman, Director of Operations Information Builders Inc., FOCUS Division August, 2008.

Documents