Am I liable for Nanny taxes?

We have a live in Nanny from South Africa. Originally, she entered the country with a J-1 VISA (cultural exchange) that allowed her to receive a weekly stipend for one year tax free. That was five years ago. Since that time she switched to a F-1 VISA (student) and earns money taking care of our kids and additional money under the table watching the neighbor kids. My questions is if she makes $12K to $15K per year and doesn't file Federal or State income taxes, can the IRS come after me with fines and penalties?