Reduced hard disk perfomance on 5.4 with SMP

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Reduced hard disk perfomance on 5.4 with SMP

Tarasov Alexey
Hello!

I have a server Double Xeon 2.4 GHz with Hyperthreading on.

FreeBSD version is:
> uname -a
> FreeBSD <host> 5.4-RELEASE-p7 FreeBSD 5.4-RELEASE-p7 #0: Sat Oct  8
> 11:50:02 PDT 2005    
> [hidden email]:/usr/obj/usr/src/sys/KERNEL  i386
Kernel Config:


> machine        i386
> cpu        I686_CPU
> ident        "KERNEL"
>
> # To statically compile in device wiring instead of /boot/device.hints
> #hints        "GENERIC.hints"        # Default places to look for devices.
>
> #imakeoptions    DEBUG=-g        # Build kernel with gdb(1) debug symbols
>
> #options     SCHED_ULE        # ULE scheduler
> options     SCHED_4BSD        # 4BSD scheduler
> options     PREEMPTION        # Enable kernel thread preemption
> options     INET            # InterNETworking
> options     INET6            # IPv6 communications protocols
> options     FFS            # Berkeley Fast Filesystem
> options     SOFTUPDATES        # Enable FFS soft updates support
> options     UFS_ACL            # Support for access control lists
> options     UFS_DIRHASH        # Improve performance on big directories
> options     MD_ROOT            # MD is a potential root device
> options     NFSCLIENT        # Network Filesystem Client
> options     NFSSERVER        # Network Filesystem Server
> options     NFS_ROOT        # NFS usable as /, requires NFSCLIENT
> options     MSDOSFS            # MSDOS Filesystem
> options     CD9660            # ISO 9660 Filesystem
> options     PROCFS            # Process filesystem (requires PSEUDOFS)
> options     PSEUDOFS        # Pseudo-filesystem framework
> options     GEOM_GPT        # GUID Partition Tables.
> options     COMPAT_43        # Compatible with BSD 4.3 [KEEP THIS!]
> options     COMPAT_FREEBSD4        # Compatible with FreeBSD4
> #options     COMPAT_FREEBSD5        # Compatible with FreeBSD5
> options     SCSI_DELAY=5000        # Delay (in ms) before probing SCSI
> options     KTRACE            # ktrace(1) support
> options     SYSVSHM            # SYSV-style shared memory
> options     SYSVMSG            # SYSV-style message queues
> options     SYSVSEM            # SYSV-style semaphores
> options     _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time
> extensions
> options     KBD_INSTALL_CDEV    # install a CDEV entry in /dev
> options     AHC_REG_PRETTY_PRINT    # Print register bitfields in debug
>                     # output.  Adds ~128k to driver.
> options     AHD_REG_PRETTY_PRINT    # Print register bitfields in debug
>                     # output.  Adds ~215k to driver.
> options     ADAPTIVE_GIANT        # Giant mutex is adaptive.
>
> # Debugging for use in -current
> #options     KDB            # Enable kernel debugger support.
> #options     DDB            # Support DDB.
> #options     GDB            # Support remote GDB.
> #options     INVARIANTS        # Enable calls of extra sanity checking
> #options     INVARIANT_SUPPORT    # Extra sanity checks of internal
> structures, required by INVARIANTS
> #options     WITNESS            # Enable checks to detect deadlocks
> and cycles
> #options     WITNESS_SKIPSPIN    # Don't run witness on spinlocks for
> speed
>
>
> #Added options
>
>
> device        pf              # OpenBSD PacketFilter
> device         pflog
> device         pfsync
>
> options         ALTQ
> options         ALTQ_CBQ        # Class Bases Queuing (CBQ)
> options         ALTQ_RED        # Random Early Detection (RED)
> options         ALTQ_RIO        # RED In/Out
> options         ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
> options         ALTQ_PRIQ       # Priority Queuing (PRIQ)
> options         ALTQ_NOPCC      # Required for SMP build
>
> options         NMBCLUSTERS=16384
> options        NETGRAPH
>
> #End
>
>
> # To make an SMP kernel, the next two lines are needed
> options     SMP            # Symmetric MultiProcessor Kernel
> device        apic            # I/O APIC
>
> # Bus support.  Do not remove isa, even if you have no isa slots
> device        isa
> device        eisa
> device        pci
>
> # Floppy drives
> device        fdc
>
> # ATA and ATAPI devices
> device        ata
> device        atadisk        # ATA disk drives
> device        ataraid        # ATA RAID drives
> device        atapicd        # ATAPI CDROM drives
> device        atapifd        # ATAPI floppy drives
> device        atapist        # ATAPI tape drives
> options     ATA_STATIC_ID    # Static device numbering
>
> # SCSI Controllers
> device        ahb        # EISA AHA1742 family
> device        ahc        # AHA2940 and onboard AIC7xxx devices
> device        ahd        # AHA39320/29320 and onboard AIC79xx devices
> device        amd        # AMD 53C974 (Tekram DC-390(T))
> device        isp        # Qlogic family
> #device     ispfw        # Firmware for QLogic HBAs- normally a module
> device        mpt        # LSI-Logic MPT-Fusion
> #device        ncr        # NCR/Symbios Logic
> device        sym        # NCR/Symbios Logic (newer chipsets + those
> of `ncr')
> device        trm        # Tekram DC395U/UW/F DC315U adapters
>
> device        adv        # Advansys SCSI adapters
> device        adw        # Advansys wide SCSI adapters
> device        aha        # Adaptec 154x SCSI adapters
> device        aic        # Adaptec 15[012]x SCSI adapters, AIC-6[23]60.
> device        bt        # Buslogic/Mylex MultiMaster SCSI adapters
>
> device        ncv        # NCR 53C500
> device        nsp        # Workbit Ninja SCSI-3
> device        stg        # TMC 18C30/18C50
>
> # SCSI peripherals
> device        scbus        # SCSI bus (required for SCSI)
> device        ch        # SCSI media changers
> device        da        # Direct Access (disks)
> device        sa        # Sequential Access (tape etc)
> device        cd        # CD
> device        pass        # Passthrough device (direct SCSI access)
> device        ses        # SCSI Environmental Services (and SAF-TE)
>
> # RAID controllers interfaced to the SCSI subsystem
> device        amr        # AMI MegaRAID
> device        arcmsr        # Areca SATA II RAID
> device        asr        # DPT SmartRAID V, VI and Adaptec SCSI RAID
> device        ciss        # Compaq Smart RAID 5*
> device        dpt        # DPT Smartcache III, IV - See NOTES for options
> device        hptmv        # Highpoint RocketRAID 182x
> device        iir        # Intel Integrated RAID
> device        ips        # IBM (Adaptec) ServeRAID
> device        mly        # Mylex AcceleRAID/eXtremeRAID
> device        twa        # 3ware 9000 series PATA/SATA RAID
>
> # RAID controllers
> device        aac        # Adaptec FSA RAID
> device        aacp        # SCSI passthrough for aac (requires CAM)
> device        ida        # Compaq Smart RAID
> device        mlx        # Mylex DAC960 family
> device        pst        # Promise Supertrak SX6000
> device        twe        # 3ware ATA RAID
>
> # atkbdc0 controls both the keyboard and the PS/2 mouse
> device        atkbdc        # AT keyboard controller
> device        atkbd        # AT keyboard
> device        psm        # PS/2 mouse
>
> device        vga        # VGA video card driver
>
> device        splash        # Splash screen and screen saver support
>
> # syscons is the default console driver, resembling an SCO console
> device        sc
>
> # Enable this for the pcvt (VT220 compatible) console driver
> #device        vt
> #options     XSERVER        # support for X server on a vt console
> #options     FAT_CURSOR    # start with block cursor
>
> device        agp        # support several AGP chipsets
>
> # Floating point support - do not disable.
> device        npx
>
> # Power management support (see NOTES for more options)
> #device        apm
> # Add suspend/resume support for the i8254.
> device        pmtimer
>
> # PCCARD (PCMCIA) support
> # PCMCIA and cardbus bridge support
> device        cbb        # cardbus (yenta) bridge
> device        pccard        # PC Card (16-bit) bus
> device        cardbus        # CardBus (32-bit) bus
>
> # Serial (COM) ports
> device        sio        # 8250, 16[45]50 based serial ports
>
> # Parallel port
> device        ppc
> device        ppbus        # Parallel port bus (required)
> device        lpt        # Printer
> device        plip        # TCP/IP over parallel
> device        ppi        # Parallel port interface device
> #device        vpo        # Requires scbus and da
>
> # If you've got a "dumb" serial or parallel PCI card that is
> # supported by the puc(4) glue driver, uncomment the following
> # line to enable it (connects to the sio and/or ppc drivers):
> #device        puc
>
> # PCI Ethernet NICs.
> device        de        # DEC/Intel DC21x4x (``Tulip'')
> device        em        # Intel PRO/1000 adapter Gigabit Ethernet Card
> device        ixgb        # Intel PRO/10GbE Ethernet Card
> device        txp        # 3Com 3cR990 (``Typhoon'')
> device        vx        # 3Com 3c590, 3c595 (``Vortex'')
>
> # PCI Ethernet NICs that use the common MII bus controller code.
> # NOTE: Be sure to keep the 'device miibus' line in order to use these
> NICs!
> device        miibus        # MII bus support
> device        bfe        # Broadcom BCM440x 10/100 Ethernet
> device        bge        # Broadcom BCM570xx Gigabit Ethernet
> device        dc        # DEC/Intel 21143 and various workalikes
> device        fxp        # Intel EtherExpress PRO/100B (82557, 82558)
> device        lge        # Level 1 LXT1001 gigabit Ethernet
> device        nge        # NatSemi DP83820 gigabit Ethernet
> #device        nve        # nVidia nForce MCP on-board Ethernet Networking
> device        pcn        # AMD Am79C97x PCI 10/100(precedence over 'lnc')
> device        re        # RealTek 8139C+/8169/8169S/8110S
> device        rl        # RealTek 8129/8139
> device        sf        # Adaptec AIC-6915 (``Starfire'')
> device        sis        # Silicon Integrated Systems SiS 900/SiS 7016
> device        sk        # SysKonnect SK-984x & SK-982x gigabit Ethernet
> device        ste        # Sundance ST201 (D-Link DFE-550TX)
> device        ti        # Alteon Networks Tigon I/II gigabit Ethernet
> device        tl        # Texas Instruments ThunderLAN
> device        tx        # SMC EtherPower II (83c170 ``EPIC'')
> device        vge        # VIA VT612x gigabit Ethernet
> device        vr        # VIA Rhine, Rhine II
> device        wb        # Winbond W89C840F
> device        xl        # 3Com 3c90x (``Boomerang'', ``Cyclone'')
>
> # ISA Ethernet NICs.  pccard NICs included.
> device        cs        # Crystal Semiconductor CS89x0 NIC
> # 'device ed' requires 'device miibus'
> device        ed        # NE[12]000, SMC Ultra, 3c503, DS8390 cards
> device        ex        # Intel EtherExpress Pro/10 and Pro/10+
> device        ep        # Etherlink III based cards
> device        fe        # Fujitsu MB8696x based cards
> device        ie        # EtherExpress 8/16, 3C507, StarLAN 10 etc.
> device        lnc        # NE2100, NE32-VL Lance Ethernet cards
> device        sn        # SMC's 9000 series of Ethernet chips
> device        xe        # Xircom pccard Ethernet
>
> # ISA devices that use the old ISA shims
> #device        le
>
> # Wireless NIC cards
> device        wlan        # 802.11 support
> device        an        # Aironet 4500/4800 802.11 wireless NICs.
> device        awi        # BayStack 660 and others
> #device        ral        # Ralink Technology RT2500 wireless NICs.
> device        wi        # WaveLAN/Intersil/Symbol 802.11 wireless NICs.
> #device        wl        # Older non 802.11 Wavelan wireless NIC.
>
> # Pseudo devices.
> device        loop        # Network loopback
> device        mem        # Memory and kernel memory devices
> device        io        # I/O device
> device        random        # Entropy device
> device        ether        # Ethernet support
> device        sl        # Kernel SLIP
> device        ppp        # Kernel PPP
> device        tun        # Packet tunnel.
> device        pty        # Pseudo-ttys (telnet etc)
> device        md        # Memory "disks"
> device        gif        # IPv6 and IPv4 tunneling
> device        faith        # IPv6-to-IPv4 relaying (translation)
>
> # The `bpf' device enables the Berkeley Packet Filter.
> # Be aware of the administrative consequences of enabling this!
> # Note that 'bpf' is required for DHCP.
> device        bpf        # Berkeley packet filter
>
> # USB support
> device        uhci        # UHCI PCI->USB interface
> device        ohci        # OHCI PCI->USB interface
> device        ehci        # EHCI PCI->USB interface (USB 2.0)
> device        usb        # USB Bus (required)
> #device        udbp        # USB Double Bulk Pipe devices
> device        ugen        # Generic
> device        uhid        # "Human Interface Devices"
> device        ukbd        # Keyboard
> device        ulpt        # Printer
> device        umass        # Disks/Mass storage - Requires scbus and da
> device        ums        # Mouse
> #device        ural        # Ralink Technology RT2500USB wireless NICs
> device        urio        # Diamond Rio 500 MP3 player
> device        uscanner    # Scanners
> # USB Ethernet, requires miibus
> device        aue        # ADMtek USB Ethernet
> device        axe        # ASIX Electronics USB Ethernet
> device        cdce        # Generic USB over Ethernet
> device        cue        # CATC USB Ethernet
> device        kue        # Kawasaki LSI USB Ethernet
> device        rue        # RealTek RTL8150 USB Ethernet
>
> # FireWire support
> device        firewire    # FireWire bus code
> device        sbp        # SCSI over FireWire (Requires scbus and da)
> device        fwe        # Ethernet over FireWire (non-standard!)
Hard disk - RAID array 0+1.

Operations with hard disk (e.g. unpacking huge .tgz archive with a lot
of files) are very slow. Why it can be?

--
Best regards,
Tarasov Alexey.

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Reduced hard disk perfomance on 5.4 with SMP

Kris Kennaway
On Sat, Nov 12, 2005 at 10:50:03PM +0300, Tarasov Alexey wrote:
> Hello!
>
> I have a server Double Xeon 2.4 GHz with Hyperthreading on.

> Operations with hard disk (e.g. unpacking huge .tgz archive with a lot
> of files) are very slow. Why it can be?

VFS was Giant-locked on 5.4, and there is contention between multiple
processes (e.g. tar and bufdaemon).  Try 6.0 instead.

Kris

attachment0 (194 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Reduced hard disk perfomance on 5.4 with SMP

Tarasov Alexey
Kris Kennaway wrote:
>
> VFS was Giant-locked on 5.4, and there is contention between multiple
> processes (e.g. tar and bufdaemon).  Try 6.0 instead.
>  
I've updated server to RELENG_6_0. Results of hard disk test:

> /usr/local/bin/bonnie
> File './Bonnie.37454', size: 104857600
> Writing with putc()...done
> Rewriting...done
> Writing intelligently...done
> Reading with getc()...done
> Reading intelligently...done
> Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
>               -------Sequential Output-------- ---Sequential Input--
> --Random--
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block---
> --Seeks---
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  
> /sec %CPU
>           100  3733  3.8  3112  1.0  2691  1.5 105299 99.9 1451514
> 100.0 61287.7 180.5


Compare with my home computer results:

> bonnie
> File './Bonnie.36180', size: 104857600
> Writing with putc()...done
> Rewriting...done
> Writing intelligently...done
> Reading with getc()...done
> Reading intelligently...done
> Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
>               -------Sequential Output-------- ---Sequential Input--
> --Random--
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block---
> --Seeks---
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  
> /sec %CPU
>           100 47381 32.2 48026  9.2 44852  8.7 83843 68.5 1184280 78.9
> 84180.4 82.5

--
Best regards,
Tarasov Alexey.

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[hidden email]"