SQL’izing Crystal Reports · 2018. 4. 14. · To make sure that both Sage 300 and Crystal speak a common language ODBC uses SQL (Structured Query Language). WARNING: SQL is commonly

Session Title - 1

{Session Number(6-5, 6-6)}

{10/13/2017} 8:30AM to 11:45PM

SQL’izing Crystal Reports

Presented By:

David Hardy Progressive Reports

Session Title - 2

SQL’izing Your Crystal Reports 1. What is SQL?:

a. Structured Query Language

b. Review of Basic SQL statements

2. How Does Crystal speak to Sage 300:

a. ODBC

b. SQL Language is used.

c. Show “Show SQL” in Crystal Reports.

d. History of Crystal and SQL.

i. You used to be able to edit it directly, not anymore.

ii. You can build your own using “Add Command”.

3. Why build your own SQL:

a. Speed…Speed…Speed and more Speed!

b. Reduction in Subreports.

i. Can’t have a subreport within a subreport.

ii. Important for exports to text or Excel.

c. Complexity becomes Simplicity

i. Union Report that shows all Union Pays, Fringes and Deducts.

ii. Bringing transactions together into table.

4. Limitations:

a. Sage 300 ODBC Driver only supports a small amount of commands and functions

through Crystal/ODBC.

5. Options:

a. Anterra.

b. Sage Gateway.

c. Timberscan SQL Database.

6. Activity 1: Copy SQL from existing Crystal Report.

a. Build Report using AP Vendor and Invoice.

b. Copy the SQL from Crystal to Notepad.

c. Modify the SQL in Notepad and copy back to a new report.

7. Activity 2: Union All

a. Join Current and History GL Transactions.

8. Activity 3: Join GL Master and Prefix together.

a. Join the GL Master Account and Prefix without using a subreport.

9. Activity 4: Union Report

Session Title - 3

a. Join the Employee, Current Check, Check Deduct & Check Fringe into one report.

i. Make this into a Crosstab.

10. Conclusion…Speed, complexity and amazing ability all in writing your own SQL Reports.

What is SQL? “SQL (sometimes referred to as Structured Query Language) is a special-purpose programming language designed for managing and reporting on data in relational database management systems (RDBMS). Or, in simple terms…

“SQL is a language designed specifically for communicating with databases”

How Does Crystal Speak to Sage 300? Sage 300 does not allow most products to read or communicate directly with the software, rather it allows all other software packages out there including Excel, Word, Access and many other to read and write data to Sage 300 via ODBC. Crystal Reports fits right in with the rest of these products. When you create a Crystal Report that pulls data from Sage 300, Crystal communicates with Sage 300 via ODBC. To make sure that both Sage 300 and Crystal speak a common language ODBC uses SQL (Structured Query Language).

WARNING: SQL is commonly thought of as database software. This is incorrect, and is the source of much confusion. SQL is a language used by Database Management Systems (DBMS). The database is the container created and manipulated via the DBMS.

Advantages

SQL is not a proprietary language and is supported by almost every major database

SQL is easy to learn

SQL is a powerful

WARNING: Implementations of SQL by DBMS vendors can differ resulting in basic SQL functionality being left out or the inconsistent application of data types. Therefore, a SQL statement that works with one Database system may not work with another.

General Capabilities

Database creation (create tables, fields)

Data manipulation (create, delete, and update data)

Access control (security)

Session Title - 4

Queries (retrieving data)

SQL and ODBC ODBC (Open Database Connectivity) is a standard interface for accessing database management systems (DBMS). Therefore an application can use ODBC to query data from a DBMS, regardless of the operating system or DBMS it uses. SQL is used to interact with ODBC. Examples in this document will use ODBC to retrieve data from Sage 300 Construction and Real Estate.

SQL and Sage 300 Construction and Real Estate While there are numerous types of SQL statements, we will focus on SQL queries using the SELECT statement because this is the most useful statement to anyone other than a database programmer. SELECT statements are used to retrieve data from a database. It’s helpful to think of this in terms of rows and columns where rows represent table records and columns represent individual fields for each record that is retrieved. Keeping this idea in mind will help conceptually understand more complex query statements that can have expression results for columns with rows consolidating data from multiple database tables.

SQL Basics

While it is important to have the proper order of a SQL statement and clauses, the formatting of the statement is much less important. This includes spaces, indents, capitalization, carriage returns, blank rows, and sometimes quotes. For example, the following SQL statements all have the correct syntax and will return the same result set:

SELECT ACCOUNT, TITLE, TYPE FROM GLM_MASTER_ACCOUNT

SELECT ACCOUNT, TITLE, TYPE FROM GLM_MASTER_ACCOUNT

SELECT ACCOUNT, TITLE, TYPE FROM GLM_MASTER_ACCOUNT

select account, title, type from glm_master_account

TIP: Proper use of formatting can make a SQL statement easier to read, especially for the next person who needs to interpret the statement. Which statement above is easier to read?

Session Title - 5

The SELECT Statement

Its purpose is to retrieve information from one or more tables.

It consists of six main clauses: SELECT, FROM, WHERE, GROUP BY, HAVING, and ORDER BY.

In general, only the SELECT clause is required, however both the SELECT and FROM clauses are almost always present in all SELECT statements.

Session Title - 6

Here’s what one looks like:

SELECT Field or expression 1, Field or expression 2 FROM Table where fields exist WHERE Condition 1 AND/OR Condition 2 GROUP BY Field or expression 1, Field or expression 2 HAVING “aggregate” condition 1 AND/OR “aggregate” condition 2 ORDER BY Field or expression 1, Field or expression 2

The SELECT..FROM Clauses PURPOSE: To return one or more fields and/or expressions from one or more tables SELECT and FROM are two different clauses, but we will treat them as one since they are almost required together.

EXAMPLE 1 - to retrieve one column of data, in this case the account number from the GL Account table:

SELECT

Account

FROM

GLM_MASTER__ACOUNT

“Select list”

Result set based on the Select Statement

Session Title - 7

TIP: Using an asterisk (*) in the select list indicates that all columns of a table should be included in the result set. For example, the following statement will return all fields in the General Ledger Account Table:

SELECT * FROM GLM_MASTER__ACOUNT

The SELECT..FROM Clauses (continued)

EXAMPLE 2 to retrieve two columns of data, in this case the account number and account description from the GL Account table:

SELECT

Account , Account_Title

FROM

GLM_MASTER__ACCOUNT

NOTE: Not all applications and ODBC drivers display the field and table names the same. Sometimes the field name will have the table name preceding it.

The following is another way to view the SQL statement in Example 2:

SELECT

GLM_MASTER__ACCOUNT.Account,

GLM_MASTER__ACCOUNT.Account_Title

FROM

GLM_MASTER__ACCOUNT GLM_MASTER__ACCOUNT

When more than one field is being retrieved in the

SELECT statement, all fields are separated by commas

except for the last one

Session Title - 8

The WHERE Clause PURPOSE: Used to filter the result set form a SELECT statement. Any expression written in this section must return a “true” or “false” (Boolean). The WHERE clause must come after the SELECT..FROM clauses and before any other clauses. EXAMPLE 3 – Select account, account title, and account type from the GLM_Account table and filter the data to only show records where the account title contains ‘cash.’

SELECT Account, Account_Title AS Title, Account_Type AS Type FROM GLM_MASTER__ACOUNT WHERE Account_Title LIKE '%Cash%'

TIP: The LIKE condition allows you to use wildcards in the WHERE clause of an SQL statement. This allows you to perform pattern matching.

The patterns that you can choose from are:

% allows you to match any string of any length (including zero length)

_ allows you to match on a single character

Using AS “column name” will customize column

headers

Always include text in SINGLE quotes. See tip below for info on the LIKE condition

Session Title - 9

The WHERE Clause (continued) Example 4 – Take the same fields in example 3, but this time filter it for records where the first two characters in the account field is either 21 or 50 and the first letter in the account title is ‘A’ and the account type is ‘Current assets.’

SELECT Account, Account_Title AS Title, Account_Type AS Type FROM GLM_MASTER__ACOUNT WHERE Left(Account,2) in ('21','50') and left(Account_Title,1)='A' and Account_Type = 'Current assets'

NOTE: Multiple conditions may be combined in the WHERE clause by using the AND and OR operators.

NOTE: WHERE clause operators include:

= Equality

<> Non-equality

< Less than

<= Less than or equal to

> Greater than

>= Greater than or equal to

IS NULL Is a NULL value

IN Is one of a series of values

The result of the WHERE clause for any given record must be a “true” or “false”

(Boolean).

Always include text in SINGLE quotes. See tip below for info on the LIKE condition

Session Title - 10

The ORDER BY Clause EXAMPLE 5 – Sort the fields in EXAMPLE 5 by account title and then account in ascending order. SELECT Account AS Account, Account_Title AS Account_Name, Account_Type AS Type FROM GLM_MASTER__ACCOUNT WHERE left(Account,2) in ('21','50') and left(Account_Title,1)='A' and Account_Type = 'Current assets' ORDER BY 2,1

TIP: Column numbers may be used to specify the sort order instead of field names. This is especially useful when formulas are used to retrieve column fields.

Select is like the

fields picked from

Field Explorer

From is like the

tables in Database

Expert.

Where is nothing more

than Select Expert and

Order by is sorting.

Session Title - 11

The GROUP BY Clause PURPOSE: The GROUP BY clause is used in conjunction with aggregate functions to summarize the result set of a SELECT statement. Examples of aggregate functions are:

AVG() Returns the column’s average value COUNT() Returns the number of rows in a column MAX() Returns the column’s highest value MIN() Returns the column’s lowest value SUM() Returns the sum of a columns values

The GROUP BY clause must come after the SELECT..FROM and WHERE clauses and before any others. EXAMPLE 7 – Show the total amount of open invoices for each vendor for which invoices have been recorded

SELECT Vendor AS Vendor_ID, Sum(Amount) AS Invoice_Amount FROM APM_MASTER__INVOICE WHERE Status = 'Open' GROUP BY Vendor

TIP: Fields used in the WHERE clause do not need to be included in the select list. The status field used in the WHERE clause in example 7 is not one of the two fields listed under SELECT.

Aggregate function. This one will return a sum of the invoice amount field for each vendor (as specified in

the GROUP BY clause)

Session Title - 12

The GROUP BY Clause (continued) EXAMPLE 8 – Add a column for the number of open invoices to example 7.

SELECT Vendor, Sum(Amount) AS Invoice_Amount Count(Invoice) AS Number of invoices FROM APM_MASTER__INVOICE WHERE Status = 'Open' GROUP BY Vendor

WARNING: Aggregate functions like SUM() and COUNT() in the example above cannot be used in a SELECT statement without the GROUP BY clause.

Aggregate function. This one will return a sum of the invoice amount field for each vendor (as specified in

the GROUP BY clause)

Session Title - 13

SELECT Clause Order Overview

Clause Description Required Crystal Equivalent Select Data to be returned Yes Fields selected in Field Explorer

(Columns)

From Tables to retrieve from Yes Database Expert and linking

Where Record Level Filtering No Select Expert (Conditions)

Group By Group summarization No Group Expert (Conditions on Group Totals)

Having Group Level Summarization No Group Level Select Expert

Order By Output Sort Order No Sort Order Expert

Session Title - 14

Activity 1 Copy SQL from Existing Report 1. Create a new Report with the AP_MASTER_VENDOR AND AP_MASTER_INVOICE tables are

used.

2. Group on Vendor ID on the Invoice table.

3. Select the Invoice, Description, Status, Amount, Amout Paid fields from the invoice and

place them in the details section.

4. From the Database menu, select “Show SQL”.

5. Highlight the text and copy the SQL statement to your clipboard.

6. Paste the SQL to notepad or WordPad.

Session Title - 15

7. Inside Notepad copy “APM_MASTER__INVOICE”.”Amount” to a new sentence directly

below the Amount Paid sentence. Once pasted, change the word Amount to Retainage.

8. Once done your SQL will look like this:

9. Start a new Report from scratch.

10. Select the table called “ADD Command”.

11. Paste the new SQL back into the empty box. Then Click OK.

12. Drag all the fields from your new table “command” into the details section and print

preview.

a. You should notice that all the fields you had in the original report are still there but

now you also have “Retainage”.

Session Title - 16

This report was simply to show you that you can copy the SQL from an existing report, modify it and start a new report with that modified SQL. You cannot copy it back to an existing report that was built the “normal” way using Database Expert, pulling in tables. Save your work as Activity 1A.rpt .

Session Title - 17

Activity 2 Create a GL Report with Current and History combined One of the great features of the original Sage 300 Report Designer is the ability to combine records from both Current and History. Crystal however only allows you to report on Current “or” History. Unless of course we write our own SQL that will combine the two tables. So let’s do that!

1. Open Crystal Reports and create a new Report with ONLY the GLT_Current__Transaction.

2. Select Account, Trans Description, Debit, Credit & Batch. Place each of these items in the

detail section.

a. Print Preview and make note of how many transactions print.

b. Go to the Database Menu – Show SQL.

c. Copy your SQL to notepad.

d. In Note pad change all occurrences of the word “Current” to “History”

e. Push the entire paragraph into notepad down four lines.

f. Then paste the original SQL into notepad at the top of the page.

g. Between the original sql and the modified sql that has History instead of Current

type the words “union all”. See example below:

Session Title - 18

Notice that the SQL for current and history are identical. When you need to merge two tables you need to have the exact named fields in both sections. Union All basically means combine the data from both and return the data to Crystal as one table instead of two separate tables.

3. Copy the SQL from notepad and paste it into a new report where you selected “Add

Command” as your table.

4. Using field explorer select Account, Trans Description, Debit, Credit & Batch. Place each

of these items in the detail section.

5. Print Preview and notice that you now have more transactions visible in Print Preview as it

is now combining Current and History into one report.

6. Save the report as Activity 2.rpt.

Session Title - 19

Activity 3 Create a GL Account & Prefix Report In the past when creating a Crystal GL Report that included the Prefix table you needed a subreport to pull in that prefix name and other fields from that table. Now we can bring them in all in one SQL statement. While doing this lets use an alias to simply the code. Review the code below: select a.Account, a.Account_Title, a.Account_Type, a.Current_Balance, p. Account_Prefix_A, p.Account_Prefix_A_Description, p.Period_Code as CurrentMonth, p.Period_Ending_Date from GLM_MASTER__ACCOUNT a inner join GLM_MASTER__ACCOUNT_PREFIX_A p on left(a.Account,2) = p.Account_Prefix_A

1. Start a new report and select “Add Command”.

a. In the open blank box type the SQL you see above.

b. Take special attention to notice that the second “_” in the From statement is

actually two “__” not one “_”.

2. Click OK. If you don’t get an error you’ve done everything correctly.

3. Group on Account Prefix A.

a. Include the prefix Description in the group header.

b. Place the Account, Title, Account Type and Current Balance in the detail section.

Notice that you have now created a report that includes information from both Account and Prefix without needing a Subreport. Not only will this report run faster but it will also allow you to group and filter on fields on the Prefix table that you could not have done before. We use this as the starting point for a new financial statement with drill down capability. Without using subreports and the ability to pull in transactions and budgets makes writing complex financials much easier.

Session Title - 20

Activity 4 Create a PR Union Report I’ve often been asked how to build a report that will print for each employee all the Union Pays, Fringes and deducts for Union reporting needs. In the past it would often include the use of subreports. However, these could be slow, cumbersome to build and almost always made it impossible to export to Excel or text files. For our last activity we will build a SQL statement that will combine all this information into one straight forward report that can be easily formatted anyway you want it. Today we will be build a cross tab (similar to Excel Pivot Table). Let’s review the code below: select t.Employee, t.Period_End_Date, t.Union_ID, t.Union_Local, t.Union_Class, t.Pay_ID, sum(t.Amount)as Amount, SUM(t.Units) as Hrs, e.Employee_Name, 'Pay' as Type, t.Pay_Type as PyTp, t.Pay_ID as Split FROM PRT_CURRENT__TIME t inner join PRM_MASTER__EMPLOYEE e on t.Employee = e.Employee where t.Union_ID <> '' group by t.Employee,t.Period_End_Date,t.Pay_ID,t.Union_ID,t.Union_Local,t.Union_Class,e.Employee_Name,t.Pay_Type union all

Session Title - 21

select f.Employee, f.Period_End_Date, f.Union_ID, f.Union_Local, f.Union_Class, f.Fringe_ID as Pay_ID, sum(f.Amount)as Amount, 0.00 as Hrs, fe.Employee_Name, 'Fringe' as Type, f.Employee as PyTp, f.Fringe_ID as Split from PRT_CURRENT__CHECK_FRINGE f inner join PRM_MASTER__EMPLOYEE fe on f.Employee = fe.Employee where f.Union_ID <> '' group by f.Employee,f.Period_End_Date,f.Fringe_ID,f.Union_ID,f.Union_Local,f.Union_Class,fe.Employee_Name,f.Employee union all select d.Employee, d.Period_End_Date, d.Union_ID, d.Union_Local, d.Union_Class, d.Deduction_ID as Pay_ID, sum(d.Amount)as Amount, 0.00 as Hrs, de.Employee_Name, 'Deduct' as Type, d.Employee as PyTp, d.Deduction_ID as split from PRT_CURRENT__CHECK_DEDUCT d inner join PRM_MASTER__EMPLOYEE de on d.Employee = de.Employee where d.Union_ID <> '' group by d.Employee,d.Period_End_Date,d.Deduction_ID,d.Union_ID,d.Union_Local,d.Union_Class,de.Employee_Name,d.Employee

1. Type the exact code above into a new blank report by selecting the table “Add Command”

Session Title - 22

2. If you type it all correctly when you hit OK, you’ll have one large table with all the information in your

report.

3. From the Insert Menu in Crystal select “Cross Tab”.

a. Place the cross tab in the Report Header.

b. Right click on the Cross table and select “Cross Tab Expert”.

c. Fill in the cross tab just like the picture below:

d. Click OK.

Session Title - 23

4. Print Preview the report.

5. Right click on the Cross Tab again, chose “Cross Tab Expert”.

a. Select your desired formatting options from the Style and Customize Style tabs.

6. Save your Report as Activity 4.

Session Title - 24

Summary

SELECT t.Job, right(t.Job,4) as Base, j.Project_Manager, t.Cost_Code, t.Category, t.Transaction_Type, t.Transaction_Date, t.Accounting_Date, t.Period_End_Date, t.Application_of_Origin, t.Units, t.Amount, cc.Description as CC_desc, cc.JTD_Production_Units, cc.MTD_Production_Units, cc.Orig_Production_Units_Est, cc.Revised_Prodctn_Units_Est, cc.Production_Unit_Desc, cc.Cost_At_Comp, c.Description as Cat_desc, j.Description as Job_Desc

FROM JCT_CURRENT__TRANSACTION t

INNER JOIN JCM_MASTER__JOB j On t.Job=j.Job

LEFT OUTER JOIN JCM_MASTER__COST_CODE cc ON t.Job=cc.Job AND

t.Cost_Code=cc.Cost_Code

LEFT OUTER JOIN JCM_MASTER__CATEGORY c

ON t.Job=c.Job AND

t.Cost_Code=c.Cost_Code AND

Session Title - 25

t.Category=c.Category

WHERE t.Transaction_Type in ('AP cost' ,'EQ cost','JC cost', 'PR cost','IV cost', 'SM cost','Original estimate', 'Approved est changes', 'Estimated prod units' ,'Aprvd prod unt chngs','Prod units in place') and left(t.Job,6) <> '000000' and

t.Transaction_Date>= {?Start Date} and

(

right(t.Job,4) LIKE Case WHEN '{?Job No.}' = '' THEN '%%%%' else '{?Job No.}' END

)

AND

(

upper(j.Project_Manager) LIKE Case WHEN '{?Project Manager}' = '' THEN '%' else upper( '{?Project Manager}')

END

)

AND

(

j.Status LIKE Case WHEN '{?Job Status}' = 'Closed' THEN 'Closed' else 'In progress' END

)

Session Title - 26

The use of SQL Commands to write reports might at first seem a bit harder but in the long run will give you more control over your report, make your reports faster and give you report capabilities that were just not possible in the past. Because everything is passed to the server as a SQL command that the Sage 300 server fully understands, the data that is returned is only that the report can use. All you have to do is practice a little and it will be second nature. In addition these same SQL queries can be used in Excel, Access and even used with My Assistant. Thank you for attending.

David Hardy Progressive Reports PH: 971-223-3658 [email protected]