veritas32.htm 05-05-2002 Bikle ------------------------------- Installation of Veritas VM 3.2 Installing Veritas VM 3.2 boils down to completing these steps on each node: -Create the oracle user and the dba group -Add entries to /etc/system -Install these pkgs: VRTSlic VRTSvxvm VRTSvmsa -Install these pkgs: SUNWscucm SUNWudlm SUNWudlmr SUNWcvmr SUNWcvm -Install this pkg: ORCLudlm -Install the Veritas license keys with the vxlicense -c command -Run vxinstall to create rootdg and add the local disks to it. -reboot -Create oradg and add the shared disks to it. On hoot: vxdisksetup -i fabric_0 vxdisksetup -i fabric_1 vxdg -s init oradg fabric_0 vxdg -g oradg adddisk fabric_1 -Create volumes which correspond to the Oracle tablespaces I will need. On hoot: vxassist -g oradg -U gen make system 1000m vxassist -g oradg -U gen make temp 1000m vxassist -g oradg -U gen make tools 1000m vxassist -g oradg -U gen make users 1000m vxassist -g oradg -U gen make gendata 1000m vxassist -g oradg -U gen make genidx 1000m vxassist -g oradg -U gen make xml 1000m vxassist -g oradg -U gen make odm 1000m vxassist -g oradg -U gen make context 1000m vxassist -g oradg -U gen make intermedia 1000m vxassist -g oradg -U gen make olap 1000m vxassist -g oradg -U gen make example 1000m vxassist -g oradg -U gen make drsys 1000m vxassist -g oradg -U gen make cwmlite 1000m vxassist -g oradg -U gen make spare1 1000m vxassist -g oradg -U gen make spare2 1000m vxassist -g oradg -U gen make spare3 1000m vxassist -g oradg -U gen make spare4 1000m vxassist -g oradg -U gen make AredoT1 1000m vxassist -g oradg -U gen make BredoT1 1000m vxassist -g oradg -U gen make CredoT1 1000m vxassist -g oradg -U gen make AredoT2 1000m vxassist -g oradg -U gen make BredoT2 1000m vxassist -g oradg -U gen make CredoT2 1000m vxassist -g oradg -U gen make undo1I1 1000m vxassist -g oradg -U gen make undo1I2 1000m vxassist -g oradg -U gen make spfile 100m vxassist -g oradg -U gen make ctlfile 1000m vxassist -g oradg -U gen make ctlfile2 1000m vxassist -g oradg -U gen make srvcfg 500m vxedit -g oradg set user=oracle mode=644 system vxedit -g oradg set user=oracle mode=644 temp vxedit -g oradg set user=oracle mode=644 tools vxedit -g oradg set user=oracle mode=644 users vxedit -g oradg set user=oracle mode=644 gendata vxedit -g oradg set user=oracle mode=644 genidx vxedit -g oradg set user=oracle mode=644 xml vxedit -g oradg set user=oracle mode=644 odm vxedit -g oradg set user=oracle mode=644 context vxedit -g oradg set user=oracle mode=644 intermedia vxedit -g oradg set user=oracle mode=644 olap vxedit -g oradg set user=oracle mode=644 example vxedit -g oradg set user=oracle mode=644 drsys vxedit -g oradg set user=oracle mode=644 cwmlite vxedit -g oradg set user=oracle mode=644 spare1 vxedit -g oradg set user=oracle mode=644 spare2 vxedit -g oradg set user=oracle mode=644 spare3 vxedit -g oradg set user=oracle mode=644 spare4 vxedit -g oradg set user=oracle mode=644 AredoT1 vxedit -g oradg set user=oracle mode=644 BredoT1 vxedit -g oradg set user=oracle mode=644 CredoT1 vxedit -g oradg set user=oracle mode=644 AredoT2 vxedit -g oradg set user=oracle mode=644 BredoT2 vxedit -g oradg set user=oracle mode=644 CredoT2 vxedit -g oradg set user=oracle mode=644 undo1I1 vxedit -g oradg set user=oracle mode=644 undo1I2 vxedit -g oradg set user=oracle mode=644 spfile vxedit -g oradg set user=oracle mode=644 ctlfile vxedit -g oradg set user=oracle mode=644 ctlfile2 vxedit -g oradg set user=oracle mode=644 srvcfg Keep in mind that eventually the DBA will need to run the DBCA utility. The DBCA may be forced to read a .txt file which contains a mapping of Oracle objects to raw devices. The DBCA finds the .txt file via an env variable: DBCA_RAW_CONFIG=rawdevices.txt export DBCA_RAW_CONFIG A demo of rawdevices.txt: control1=/dev/vx/rdsk/oradg/ctlfile control2=/dev/vx/rdsk/oradg/ctlfile2 cwmlite=/dev/vx/rdsk/oradg/cwmlite drsys=/dev/vx/rdsk/oradg/drsys example=/dev/vx/rdsk/oradg/example indx=/dev/vx/rdsk/oradg/genidx odm=/dev/vx/rdsk/oradg/odm redo1_1=/dev/vx/rdsk/oradg/AredoT1 redo1_2=/dev/vx/rdsk/oradg/BredoT1 redo2_1=/dev/vx/rdsk/oradg/AredoT2 redo2_2=/dev/vx/rdsk/oradg/BredoT2 spfile=/dev/vx/rdsk/oradg/spfile system=/dev/vx/rdsk/oradg/system temp=/dev/vx/rdsk/oradg/temp tools=/dev/vx/rdsk/oradg/tools undotbs1=/dev/vx/rdsk/oradg/undo1I1 undotbs2=/dev/vx/rdsk/oradg/undo1I2 users=/dev/vx/rdsk/oradg/users xml=/dev/vx/rdsk/oradg/xml Also, it is a good idea to check that Oracle could write/read to/from the the raw devices. I used the /bin/dd utility to make this check. Here is a mini screen dump: hoot% /bin/dd if=/bikle/jdev9i_902_base.zip of=AredoT1 bs=2048 write: Invalid argument 28962+1 records in 28962+1 records out hoot% /bin/dd if=AredoT1 of=/home0/oracle/bikle/jdev9i_902_base.zip bs=2048 512000+0 records in 512000+0 records out hoot% In situations when I need to reboot both nodes, it's a good idea to reboot them serially rather than at the same time. The cluster has the best chance of staying healthy if at least one node is in a normal state. If all the nodes are going through a simultaneous reboot we face a small risk of cluster confusion setting in. One rookie mistake I made was that I rebooted both nodes before I ran vxinstall. This mistake prevented the nodes from coming up cleanly. Here is some text which appeared in my console window: vxvm:vxdctl: ERROR: vxconfigd is not running, cannot enable Aug 31 14:34:17 hoot savecore: reboot after panic: Failfast: Aborting because "ucmmd" died 30 seconds ago. I fixed this mistake by gaining access to the console via a telnet session into a console terminal server. Once the session was established, I typed ^] which dropped me into a prompt which looked like this: telnet> I typed this: send break It responded with this: ok I typed: boot -x which allowed the node to boot in a more primative state. Once the node booted, I was then allowed to run vxinstall. Here is a vxinstall screen dump: vxinstallScreenDump.txt Another mistake I made was that I added the shared disks to the rootdg of both nodes. Due to this mistake, it is now easy for me to remember that the shared disks belong in the oradg not the rootdg. The mistake manifested itself during a reboot; the nodes failed to come up cleanly after sending out an error which looked like this: vxvm:vxdctl: ERROR: vxconfigd is not running, cannot enable Aug 31 14:34:17 hoot savecore: reboot after panic: Failfast: Aborting because "ucmmd" died 30 seconds ago.