ORA-03137: TTC protocol internal error : [12333]错误一例

Comments

4 responses to “ORA-03137: TTC protocol internal error : [12333]错误一例”

1. 

   maclean

   September 5, 2010

   ora-600 [12333] / ora-3137 [12333] Troubleshooting
   Applies to:
   Oracle Server – Enterprise Edition – Version: 10.2.0.1 to 11.1.0.7 – Release: 10.2 to 11.1
   Information in this document applies to any platform.

   Symptoms
   ora-600 [12333]  / ora-3137 [12333]
   Cause
   This could happens if a statement take a time and the process could have a timeout and died , this errors means that the process died or was killed

   Solution
   The ora-600 [12333] error is only telling us that the client and server process communication was terminated due to some event.
   This error is only affecting the session seeing the error and not the instance as a whole.

   For any ONE-TIME occurrence we can safely ignore the error.
   This error does not relate to any potential data corruption or any data loss.
   If it would come back and re-occur over and over, would indicate a problem, but isolated  incident is not a concern.

   Please first check for any incompatibility between database version and clients, see :
   Note 207303.1 –  Client / Server / Interoperability Support Between Different Oracle Versions

   Then we will need to to check if we are doing any remote processing to monitor the process and the network traffic.

   We should look for which session/program/application that triggers this issue.
   This shall be found from the trace files under the session state object ‘(session)’.

   Sometimes the ORA-00600 [12333] trace files will not capture the session information, in such cases we can Check with the end users about any issues around the time-frame of this internal error.

   There is a HIGH chance for application run-time exception to lead to this ORA-00600 [12333].
   Due to the unhandled exceptions there is a chance for incomplete information passed to the server, hence the internal errors are raised. In such cases, handling those exceptions (or) addressing those errors on the application code will resolve the issue.

   In few cases, the problem application will work for a while before it terminates with the error.
   This could be due to the TIMEOUT settings at the Application Server (or) Forms Server.,etc.
   To resolve this either tune the failing application to complete within the TIMEOUT value (or) reset the TIMEOUT to an optimal value.

   There could be more possibilities here; in simple words the approach should be from application side, which would help us to understand the other issues that, triggered this network interruption.

   In-case if all the above suggestions does not help to identify the cause, and the ORA-00600 [12333] persists we may need to enable the networking tracings from the problem client and need to review the network traces to understand where the problem occurs. See:
   Note 219968.1 – SQL*Net, Net8, Oracle Net Services – Tracing and Logging at a Glance

   Check also the following note for more info on already reported issues :
   Note 35928.1 – ORA-600 [12333] "Fatal Two-Task Protocol Violation"

   In case, you notice JDBC Client as the failing program under Session state object, please check and ensure that you use latest compatible version of JDBC Drivers in your application side.
   You can refer to :
   Note 430839.1 To verify the version of JDBC Drivers and for Installation steps

   In case jdbc is used, then a good procedure to prevent this kind of behavior is to control the following things from the jdbc application side :

   – Check that you are using the last jdbc driver
   – Monitor and try to handle correctly the timeouts from the application
   – Check that idle or died programs/processes are not just killed at the client side but using a good cleaning procedure.
   – Increase queue sizes / buffers and similar from the jdbc side.

   An ORA-600[12333] is reported with three additional numeric values when a request is being received from a network packet and the request code in the packet is not recognized.

   Once the execution of the sql is completed, if there is any data (which is sent by the client) left on the wire which was intended to be consumed by the server during the execution of this sql ,this error occurs.

   For example, Let us say if the application is doing an array DML operation and executing the following statement "Insert into test_table ( :1 , :2)" .

   Application would have provided bind information for all the iterations. Because of some error during one of the iterations or due to some code bug if server failes to read all the bind data by the end of all the iterations this tracing will come into play and print the information "PROTOCOLVIOLATION DETECTED"

   Normally if application continues after this we would see an ORA-600[12333] error or an ORA-3106 error or some unexpected behavior.

   It is just an indication of above such conditions.

   Finally, have the network team to investigate the network for any problems. If the error is intermittent and not reproducible at will, this could indicates the problem may not be in the database, but in the environment. ORA-600 [12333] is a client communication problem involving the network and can also be caused by dropped packets.
    

   Reply
2. 

   maclean

   September 5, 2010

   ORA-600 [12333] “Fatal Two-Task Protocol Violation”
   PURPOSE:
   This article discusses the internal error “ORA-600 [12333]”, what
   it means and possible actions. The information here is only applicable
   to the versions listed and is provided only for guidance.

   ERROR:
   ORA-600 [12333] [a] [b] [c] [d]

   VERSIONS:
   Versions 6.0 to 10.2

   DESCRIPTION:

   There are three forms of the OERI(12333) error:

   i. Usage with 4 additional values: ORA-600 [PTI] [b] [c] [d]
   Reported by Oracle release 8.0 onwards

   ii. Usage with 4 additional values: ORA-600 [a] [b] [c] [d]
   Reported by Oracle release 8.0 onwards

   iii. Usage with 3 additional values: ORA-600 [a] [b] [c]
   Reported by Oracle release 7 and earlier.

   The first form of the error is “ORA-600 [12333] [PTI] [b] [c] [d]”

   This may be reported when parsing storage options on alter/create of
   various database objects. The [b], [c], [d] arguments identify a
   specific statement type being parsed.

   This form of the error indicates a problem in the sql parser. As common
   functions are used to parse the storage options of different database
   objects, this error code is used as a semantic check that the parser has
   not passed options that are invalid for the statement being parsed.

   The second form of the error is “ORA-600 [12333] [a] [b] [c] [d]”

   This represents a Two Task Protocol violation and is reported because
   an OCI call has been sent in the wrong order.

   The third form of the error is “ORA-600 [12333] [a] [b] [c]”

   The ORA-600 [12333] is reported with three additional numeric values
   when a request is being received from a network packet and the request
   code in the packet is not recognized. The three additional values report
   the invalid request values received.

   The error may have a number of different root causes. For example, a
   multi-threaded Client-side application sent an OCI call out of order, or the
   data in the network buffer may have been overwritten.

   Since there are many potential causes of this error, it is essential to
   have a reproducible testcase to correctly diagnose the underlying cause.

   ARGUMENTS:

   Form i.
   Arg [a] PTI
   Arg [b] Parse process Options
   Arg [c] Statement type
   Arg [d] Check identification, e.g., INDEX

   Form ii.
   Arg [a] TTI message
   Arg [b] OPI function code
   Arg [c] Sequence number
   Arg [d] typically 0

   Form iii.
   Arg [a] TTI Layer Function code received
   Arg [b] Function code
   Arg [c] Sequence

   FUNCTIONALITY:
   USER/ORACLE INTERFACE LAYER

   IMPACT:
   PROCESS FAILURE
   NO DATA CORRUPTION

   SUGGESTIONS:

   Update to the latest Client or Database patch set.

   Check the Known Issues section below for your version.

   Check networking connections and NLS settings: Note:77442.1.

   Reply
3. 

   admin

   September 5, 2010

   ORA-03137: TTC Protocol Internal Error : [12333] Using JDBC Driver

   Applies to:

   JDBC – Version: 11.1.0.6 and later   [Release: 11.1 and later ]
   Information in this document applies to any platform.

   Symptoms

   On 11.1.0.6 in Production, when attempting to run an application using the JDBC thin driver 11.1.0.6 against a database version 11.1.0.6, the following error occurs:

   ORA-03137: TTC protocol internal error : [12333] [10] [83] [74] [] [] [] []

   The stack trace in the database alert log is:

   ORA-03137: TTC protocol internal error : [12333] [7] [120] [108] [] [] [] []
   —– Current SQL Statement for this session (sql_id=c0czhbm4hcz98) —–
   DELETE FROM PORTAL_CONCURRENT_USER_SCRATCH
   WHERE LAST_LOGIN < :intervalStartDate

   —– Call Stack Trace —–
   calling call entry argument values in hex
   location type point (? means dubious value)
   ——————– ——– ——————– —————————-
   skdstdst()+002c bl 105e2771c
   ksedst1()+0064 bl 101fad134
   ksedst()+0028 bl ksedst1() 60000B8104C4A30 ?
   600000002050032 ?
   dbkedDefDump()+07fc bl 101fae954
   ksedmp()+0048 bl 101fad130
   opiierr()+0168 bl ksedmp() 000000000 ?
   opitsk()+1344 bl 103ec9a5c
   opiino()+09a0 bl opitsk() 110098B40 ? 000000000 ?
   opiodr()+0b48 bl 103ec7a30
   opidrv()+0440 bl opiodr() 3C61672F72 ? 41032E668 ?
   FFFFFFFFFFFF7E0 ? 073702F74 ?
   sou2o()+0090 bl opidrv() 3C05C04AEC ? 4A09A9924 ?
   FFFFFFFFFFFF7E0 ?
   opimai_real()+01b0 bl 101face8c
   main()+0090 bl opimai_real() 000000000 ? 000000000 ?
   __start()+0098 bl main() 000000000 ? 000000000 ?

   This problem does not happen if the 10.2.0.3 driver is used.

   Cause

   According to Bug 7390077 ORA-3137: TTC PROTO INTERNAL ERROR: [12333] [8] [80] [65] reported for JDBC 11.0.6.0, this bug is an RDBMS bug and not JDBC.

   Solution

   The solution is to upgrade the database to version 11.1.0.7 where the problem is fixed.

   Note: Although upgrading to database version 11.1.0.7 has been confirmed to have solved this issue in a high number of cases, error ORA-3137 [12333] accompanied by Protocol Violation messages may still occur intermittently in database versions prior to 11.2, including database 10.2.0.4 and 11.1.0.7.
   This has been reported in unpublished bug:8625762 ORA-3137 [12333] due to bind data not read from wire, in which case ORA-3137 [12333] can occur intermittently for SQL which uses user binds due to bind data being left unread on the Net connection. The bug suggests the workaround of setting “_optim_peek_user_binds”=false to avoid this problem (workaround to be used with caution as it can affect execution plans chosen by CBO), and also provides the fix by the application of Patch:8625762.

   References

   BUG:7445550 – ORA-03137: TTC PROTO INTERNAL ERROR : [12333] [21] [106] [110] [] [] [] []
   NOTE:8625762.8 – Bug 8625762 – ORA-3137 [12333] due to bind data not read from wire

    

   Reply
4. 

   admin

   September 5, 2010

   Hdr: 7445550 11.1.0.6.0 RDBMS 11.1.0.6.0 PRG INTERFACE PRODID-5 PORTID-226 ORA-3137 7390077
   Abstract: ORA-3137: TTC PROTO INTERNAL ERROR : [12333] [21] [106] [110] [] [] [] []
   PROBLEM:
   Customer is in database 11.1.0.6 and JDBC Thin driver 11.1.0.6 and is getting
   error :
   ORA-3137: TTC proto internal error : [12333] [21] [106] [110] [] [] []
   []

   There was originally bug 7148457 filed for the same customer but was closed
   as it it could not be reproduced.

   The problem still does not seem reproducable in the customers environment as
   the error occurs intermittently.

   The violations that are causing the application to fail.

   Customer is using three tier architecture.
   1 db server 11.1.0.6, 19 middle tier servers running 11.1.0.6 JDBC driver,
   and many browser clients.

   =========================
   Environment Information:

   Database: Oracle DB 11.1.0.6 – 64 bit Oracle software on 64 bit operating
   system
   JDBC Driver: Thin 11.1.0.6
   OS: x86_64 x86_64 GNU/Linux – Red Hat Enterprise Linux Server release 5
   (Tikanga)

   =========================
   RELATED BUGS:
   7148457 – same customer

   =========================
   REPRODUCIBILITY:
   No – error occurs intermittently.

   =========================
   TESTCASE:
   N/A

   =========================
   URL:
   All stack traces can be uploaded

   ========================
   WORKAROUND:
   N/A

   =========================
   STACK TRACE:

   File = XACTLY01_ora_13733.trc

   — PROTOCOL VIOLATION DETECTED —
   —– Dump Cursor sql_id=6naatzxy9kh1a xsc=0x2aaaac4af2f0 cur=0x2aaaabd41c90
   —–
   Dump Parent Cursor sql_id=6naatzxy9kh1a phd=0x44637cab8 plk=0x429f05928
   sqltxt(0x44637cab8)=select position0_.POSITION_ID as POSITION1_13_,
   position0_.VERSION as VERSION13_, position0_.NAME as NAME13_,
   position0_.DESCR as DESCR13_, position0_.INCENT_ST_DATE as INCENT5_13_,
   position0_.INCENT_END_DATE as INCENT6_13_, position0_.EFFECTIVE_START_DATE as
   EFFECTIVE7_13_, position0_.EFFECTIVE_END_DATE as EFFECTIVE8_13_,
   position0_.IS_MASTER as IS9_13_, position0_.MASTER_POSITION_ID as
   MASTER10_13_, position0_.IS_ACTIVE as IS11_13_, position0_.CREATED_DATE as
   CREATED12_13_, position0_.CREATED_BY_ID as CREATED13_13_,
   position0_.CREATED_BY_NAME as CREATED14_13_, position0_.MODIFIED_DATE as
   MODIFIED15_13_, position0_.MODIFIED_BY_ID as MODIFIED16_13_,
   position0_.MODIFIED_BY_NAME as MODIFIED17_13_, position0_.BUSINESS_ID as
   BUSINESS18_13_ from XC_POSITION position0_ where position0_.BUSINESS_ID=:1
   hash=dbb804768d9c25ae6a2959ff7c99402a
   parent=0x446379dc0 maxchild=12 plk=0x429f05928 ppn=n
   cursor instantiation=0x2aaaac4af2f0 used=1222720227 exec_id=16804067 exec=2
   child#10(0x42ee8a6c0) pcs=0x42e4f0e48
   clk=0x429f05830 ci=0x42f679438 pn=0x422ea67e8 ctx=0x43624a188
   kgsccflg=1 llk[0x2aaaac4af2f8,0x2aaaac4af2f8] idx=2b
   xscflg=c0110676 fl2=5d000009 fl3=52222008 fl4=180
   sharing failure(s)=100000000
   —– Bind Info (kkscoacd) —–
   Bind#0
   oacdty=02 mxl=22(22) mxlc=00 mal=00 scl=00 pre=00
   kxsbbbfp=2aaaacb55820 bln=22 avl=00 flg=05
   Frames pfr 0x2aaaacb55be0 siz=25528 efr 0x2aaaacb55b10 siz=25512
   kxscphp=0x2aaaabd4b900 siz=984 inu=392 nps=344
   kxscbhp=0x2aaaacb6d298 siz=984 inu=168 nps=48
   Starting SQL statement dump
   SQL Information
   user_id=30 user_name=INCENT module=JDBC Thin Client action=
   sql_id=6naatzxy9kh1a plan_hash_value=-1268437987 problem_type=4
   —– Current SQL Statement for this session (sql_id=6naatzxy9kh1a) —–
   select position0_.POSITION_ID as POSITION1_13_, position0_.VERSION as
   VERSION13_, position0_.NAME as NAME13_, position0_.DESCR as DESCR13_,
   position0_.INCENT_ST_DATE as INCENT5_13_, position0_.INCENT_END_DATE as
   INCENT6_13_, position0_.EFFECTIVE_START_DATE as EFFECTIVE7_13_,
   position0_.EFFECTIVE_END_DATE as EFFECTIVE8_13_, position0_.IS_MASTER as
   IS9_13_, position0_.MASTER_POSITION_ID as MASTER10_13_, position0_.IS_ACTIVE
   as IS11_13_, position0_.CREATED_DATE as CREATED12_13_,
   position0_.CREATED_BY_ID as CREATED13_13_, position0_.CREATED_BY_NAME as
   CREATED14_13_, position0_.MODIFIED_DATE as MODIFIED15_13_,
   position0_.MODIFIED_BY_ID as MODIFIED16_13_, position0_.MODIFIED_BY_NAME as
   MODIFIED17_13_, position0_.BUSINESS_ID as BUSINESS18_13_ from XC_POSITION
   position0_ where position0_.BUSINESS_ID=:1
   sql_text_length=806
   sql=select position0_.POSITION_ID as POSITION1_13_, position0_.VERSION as
   VERSION13_, position0_.NAME as NAME13_, position0_.DESCR as DESCR13_,
   position0_.INCENT_ST_DATE as INCENT5_13_, position0_.INCENT_END_DATE as
   INCENT6_13_, position0_.EFFECTIVE_START_DATE
   sql= as EFFECTIVE7_13_, position0_.EFFECTIVE_END_DATE as EFFECTIVE8_13_,
   position0_.IS_MASTER as IS9_13_, position0_.MASTER_POSITION_ID as
   MASTER10_13_, position0_.IS_ACTIVE as IS11_13_, position0_.CREATED_DATE as
   CREATED12_13_, position0_.CREATED_BY_ID as CRE
   sql=ATED13_13_, position0_.CREATED_BY_NAME as CREATED14_13_,
   position0_.MODIFIED_DATE as MODIFIED15_13_, position0_.MODIFIED_BY_ID as
   MODIFIED16_13_, position0_.MODIFIED_BY_NAME as MODIFIED17_13_,
   position0_.BUSINESS_ID as BUSINESS18_13_ from XC_POSITION posit
   sql=ion0_ where position0_.BUSINESS_ID=:1
   ====================== END SQL Statement Dump ======================

   *** 13:30:27.416
   —– Error Stack Dump —–
   ORA-3137: TTC protocol internal error : [1010] [] [] [] [] [] [] []
   —– Current SQL Statement for this session (sql_id=6naatzxy9kh1a) —–
   select position0_.POSITION_ID as POSITION1_13_, position0_.VERSION as
   VERSION13_, position0_.NAME as NAME13_, position0_.DESCR as DESCR13_,
   position0_.INCENT_ST_DATE as INCENT5_13_, position0_.INCENT_END_DATE as
   INCENT6_13_, position0_.EFFECTIVE_START_DATE as EFFECTIVE7_13_,
   position0_.EFFECTIVE_END_DATE as EFFECTIVE8_13_, position0_.IS_MASTER as
   IS9_13_, position0_.MASTER_POSITION_ID as MASTER10_13_, position0_.IS_ACTIVE
   as IS11_13_, position0_.CREATED_DATE as CREATED12_13_,
   position0_.CREATED_BY_ID as CREATED13_13_, position0_.CREATED_BY_NAME as
   CREATED14_13_, position0_.MODIFIED_DATE as MODIFIED15_13_,
   position0_.MODIFIED_BY_ID as MODIFIED16_13_, position0_.MODIFIED_BY_NAME as
   MODIFIED17_13_, position0_.BUSINESS_ID as BUSINESS18_13_ from XC_POSITION
   position0_ where position0_.BUSINESS_ID=:1

   —– Call Stack Trace —–
   calling call entry argument values in hex

   location type point (? means dubious value)

   ——————– ——– ——————–
   —————————-
   skdstdst()+41 call kgdsdst() 000000000 ? 000000001 ?
   7FFF9F580088 ?
   7FFF9F57EB70 ?
   000000000 ? 000000002 ?
   ksedst1()+103 call skdstdst() 000000000 ? 000000001 ?
   7FFF9F580088 ?
   7FFF9F57EB70 ?
   000000001 ? 000000002 ?
   ksedst()+39 call ksedst1() 000000000 ? 000000001 ?
   7FFF9F580088 ?
   7FFF9F57EB70 ?
   000000001 ? 000000002 ?
   dbkedDefDump()+1076 call ksedst() 000000000 ? 000000001 ?
   7FFF9F580088 ?
   7FFF9F57EB70 ?
   000000001 ? 000000002 ?
   ksedmp()+41 call dbkedDefDump() 000000003 ? 000000000 ?
   7FFF9F580088 ?
   7FFF9F57EB70 ?
   000000001 ? 000000002 ?
   opiierr()+411 call ksedmp() 000000003 ? 000000000 ?
   7FFF9F580088 ?
   7FFF9F57EB70 ?
   000000001 ? 000000002 ?
   opitsk()+5353 call opiierr() 000000003 ? 000000000 ?
   7FFF9F580088 ? 000000000 ?
   0000003F2 ? 000000001 ?
   opiino()+1026 call opitsk() 000000001 ? 000000000 ?
   000000001 ? 000000000 ?
   0000003F2 ? 000000001 ?
   opiodr()+1178 call opiino() 00000003C ? 000000004 ?
   7FFF9F587308 ? 000000000 ?
   0000003F2 ? 000000001 ?
   opidrv()+580 call opiodr() 00000003C ? 000000004 ?
   7FFF9F587308 ? 000000000 ?
   008202E40 ? 000000001 ?
   sou2o()+90 call opidrv() 00000003C ? 000000004 ?
   7FFF9F587308 ? 000000000 ?
   008202E40 ? 000000001 ?
   opimai_real()+145 call sou2o() 7FFF9F5872E0 ? 00000003C ?
   000000004 ? 7FFF9F587308 ?
   008202E40 ? 000000001 ?
   ssthrdmain()+177 call opimai_real() 000000002 ? 7FFF9F587480 ?
   000000004 ? 7FFF9F587308 ?
   008202E40 ? 000000001 ?
   main()+215 call ssthrdmain() 000000002 ? 7FFF9F587480 ?
   000000004 ? 000000000 ?
   008202E40 ? 000000001 ?
   __libc_start_main() call main() 000000002 ? 7FFF9F5875E8 ?
   +244 000000004 ? 000000000 ?
   008202E40 ? 000000001 ?
   _start()+41 call __libc_start_main() 000935D68 ? 000000002 ?
   7FFF9F5875E8 ? 000000000 ?
   008202E40 ? 000000002 ?

   There are two errors reported above:

   ORA-3137: TTC protocol internal error : [12333] [21] [106] [110] []
   ORA-3137: TTC protocol internal error : [1010] [] [] [] [] [] [] []

   The stacks are the same for both:

   skdstdst, ksedst1, ksedst, dbkedDefDump, ksedmp, opiierr,
   opitsk, opiino, opiodr, opidrv, sou2o, opimai_real, ssthrdmain,
   main, __libc_start_main, _start

   According to the TAR, the error the client application sees is:

   Exception Message : PreparedStatementCallback; uncategorized
   SQLException for SQL []; SQL state [72000];
   error code [1010]; ORA-1010: invalid OCI operation

   even though this is thin.

   Hunting for the error with opiierr finds numerous reports vs 11g. Bug
   6741577 says the ORA-3137 with [1010] happens when oracle can’t consume all
   the bytes received on the wire. This sounds to me like more data is being
   sent than expected. In that bug it suggested adding:

   event=”1010 trace name errorstack level 2″

   to the init.ora to try and get a trace from the 1010.

   Note 35928.1 says 12333 followed by 3 parameters means a request is being
   received from a network packet and the request code in the packet is not
   recognized. The three additional values report the invalid request values
   received.

   Both these issues seem to indicate some sort of corruption in the data being
   sent to the d/b, possibly because JDBC is sending bad info or there is a
   problem on the network itself. We ideally need to see a corresponding client
   and server side net trace in order to determine where the problem is
   occurring.
   There is a similar 11g bug, bug 7238542. In that bug they believe the
   problem may occur when the cursor is reloaded into the shared pool. A
   protocol violation occurs on the server but is not signalled to the client so
   the next request from the client also causes a violation which results in the
   server side aborting. Bug 7446725 was raised from this and is currently
   assigned to SE as the conclusion was a diagnostics patch would probably be
   required to resolve this.

   If the same is happening here then a workaround may be to pin the failing
   selects in the shared pool.

   Reply