TPC BENCHMARK (TM) H (779138), страница 26
Текст из файла (страница 26)
The datashould be presented as a table with one entry for each query stream executed during the Performance Test. For eachstream entry, the total elapsed time for each query in the stream and for its associated refresh functions should bereported separately to a resolution of 0.1 seconds. In addition, the minimum, maximum and average execution timefor each query and refresh function must be reported to a resolution of 0.1 seconds.8.5Availability of the Full Disclosure Report and Supporting Files Archive8.5.1The full disclosure report and supporting files archive must be readily available to the public at a reasonable charge,similar to charges for comparable documents by that test sponsor.
The report and supporting files archive must bemade available when results are made public. In order to use the phrase “TPC Benchmark H”, the full disclosurereport and supporting files archive must have been submitted electronically to the TPC using the proceduredescribed in the TPC Policies and Guidelines document.8.5.2The official full disclosure report must be available in English but may be translated to additional languages.8.6Revisions to the Full Disclosure Report and Supporting Files ArchiveRevisions to the full disclosure documentation and supporting files archive shall be handled as follows:8.6.1Substitutions will be open to challenge for a 60 day period.
No other portion of the FDR and supporting files archiveare challengeable.8.6.2During the normal product life cycle, problems will be uncovered that require changes, sometimes referred to asECOs, FCOs, patches, updates, etc. When the cumulative result of applied changes causes the QphH rating of thesystem to decrease by more than 2% from the initially reported QphH, then the test sponsor is required to re-validatethe benchmark results. The complete revision history is maintained following the query timing interval sectionshowing the revision date and description.8.6.3Full disclosure report and supporting files archive revisions may be required for other reasons according to TPCpolicies (see TPC Policy Document)TPC BenchmarkTM H Standard Specification Revision 2.17.1Page 1189: AUDITRules for auditing Pricing information are included in the TPC Pricing Specification located at www.tpc.org.When the TPC-Energy optional reporting is selected by the test sponsor, the rules for auditing of TPC-Energyrelated items are included in the TPC Energy Specification located at www.tpc.org.9.1General Rules9.1.1An independent audit of the benchmark results by a TPC certified auditor is required.
The term independent isdefined as “the outcome of the benchmark carries no financial benefit to the auditing agency other than fees earneddirectly related to the audit.” In addition, the auditing agency cannot have supplied any performance consultingunder contract for the benchmark.In addition, the following conditions must be met:a)The auditing agency cannot be financially related to the sponsor. For example, the auditing agency is financially related if it is a dependent division of the sponsor, the majority of its stock is owned by the sponsor,etc.b)The auditing agency cannot be financially related to any one of the suppliers of the measured/priced configuration, e.g., the DBMS supplier, the disk supplier, etc.9.1.2The auditor's attestation letter is to be made readily available to the public as part of the full disclosure report.
Adetailed report from the auditor is not required.9.1.3TPC-H results can be used as the basis for new TPC-H results if and only if:a)The auditor ensures that the hardware and software products are the same as those used in the prior result;b)The auditor reviews the FDR of the new results and ensures that they match what is contained in the originalsponsor's FDR;c)The auditor can attest to the validity of the pricing used in the new FDR.Comment 1: The intent of this clause is to allow a reseller of equipment from a given supplier to publish under there-seller's name a TPC-H result already published by the supplier.Comment 2: In the event that all conditions listed in Clause 9.1.3 are met, the auditor is not required to follow theremaining auditor's check list items from Clause 9.2.9.1.4Ensure that any auxiliary data structures satisfy the requirements of Clause 1.5.6.9.1.5In the event that a remote audit procedure is used in the context of a change-based audit, a remote connection to theSUT must be available for the auditor to verify selected audit items from Clause 9.2.9.2Auditor's Check List9.2.1Clause 1 Related Items9.2.1.1 Verify that the data types used for each column are conformant.
For example, verify that decimal columns can beincremented by 0.01 from -9,999,999,999.99.9.2.1.2 Verify that the tables have the required list of columns.9.2.1.3 Verify that the implementation rules are met by the test database.9.2.1.4 Verify that the test database meets the data access transparency requirements.9.2.1.5 Verify that conforming arbitrary data values can be inserted into any of the tables. Examples of verification testsTPC BenchmarkTM H Standard Specification Revision 2.17.1Page 119include:Inserting a row that is a complete duplicate of an existing row except for a distinct ‘Primary Key’ value ;Inserting a row with column values within the domain of the data type and check constraints but beyond therange of existing values.9.2.1.6 Verify that the set of auxiliary data structures (as defined in Clause 1.5.7) that exist at the end of the load test are thesame as those which exist at the end of the performance test. A similar check may be performed at any point duringthe performance test at the discretion of the auditor.Comment: The purpose of this check is to verify that no auxiliary data structures automatically generated during theperformance test may be accessed by more than one query execution.9.2.2Clause 2 Related Items9.2.2.1 Verify that the basis for the SQL used for each query is either the functional query definition or an approved variant.9.2.2.2 Verify that all SQL features used for each query, refresh functions, database loading, indexing and verificationscripts are externally documented.9.2.2.3 Verify that any deviation in the SQL from either the functional query definition or an approved variant is compliantwith the specified minor query modifications.
Verify that minor query modifications have been applied consistentlyto the set of functional query definitions or approved variants used.9.2.2.4 Verify that the executable query text produces the required output when executed against the qualification databaseusing the validation values for substitution parameters.9.2.2.5 Note the version number, release number, modification number and patch level of QGen.
Verify that the versionand release numbers match the benchmark specification.9.2.2.6 Verify that the generated substitution parameters are reasonably diverse among the streams.9.2.2.7 Verify that no aspect of the system under test, except for the database size, has changed between the demonstrationof compliance against the qualification database and the execution of the reported measurements.9.2.2.8 Verify that the refresh functions are implemented according to their definition.9.2.2.9 Verify that the transaction requirements are met by the implementation of the refresh functions.9.2.2.10 Note the method used to execute database maintenance operations9.2.2.11 Verify that the output of the validation run (Clause 2.3.1) matches the output supplied in Appendix C.9.2.3Clause 3 Related Items9.2.3.1 Verify that the required ACID properties are supported by the system under test as configured for the execution ofthe reported measurements.9.2.3.2 If one or more of the ACID tests defined in Clause 3:demonstration of support of the related ACID property.were not executed, note the rationale for waiving such9.2.3.3 Verify that SUT Power Failure has been tested as required by Clause 3.5.3 .9.2.4Clause 4 Related Items9.2.4.1 Verify that the qualification database is properly scaled and populated.TPC BenchmarkTM H Standard Specification Revision 2.17.1Page 1209.2.4.2 Verify that the test database is properly scaled.9.2.4.3 Verify that the rows in the loaded database after the performance test are correct by comparing any two files of thecorresponding Base, Insert and Delete reference data set files for each table against the corresponding rows of thedatabase.9.2.4.4 Verify that the DBGen (using the command lines provided in Appendix F) used in the benchmark generates a dataset which matches the reference data set provided in Appendix F corresponding to the scale factor used in thisbenchmark.9.2.4.5 Verify referential integrity in the database after the initial load.9.2.4.6 Verify that the qualification and test databases were constructed in the same manner so that correct behavior on thequalification database is indicative of correct behavior on the test database.9.2.4.7 Note the version number, release number, modification number and patch level of DBGen.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
